Re: Usability Test Metrics

Welcome back Lynne

The starting point for all any system measures is what the sponsors want.
The problem with this is that so few sponsors can articulate clear
targets and critical success factors. Getting product sponsors to the
point where they can agree to objective measures is extremely challenging
and requires considerable patience, political and facilitation skills.

Fraser has already suggested what I would see as a key metric for any
system.  Given that people are meant to be able to achieve something
with a system, what can they actually achieve?  Task completion and
quality metrics are thus central to most usability evaluations.

The key point is that there are no universal usability metrics.  Agreeing
on what is important and how to measure it is central to the usability
specialist's relationship with project sponsors.  Ideally, these will be
in place *before* design commences, but this is usually wishful thinking.

I see the key skill of HCI specialists as being able to advise on
appropriate metrics, to be able to choose design features etc. that have
the best chance of meeting product objectives, and being able to design,
implement and interpret user testing based on these metrics.  In this
sense, one cannot do user-centred design without having some performance
or preference targets.

In short, you can't ignore metrics.  If you ignore them before testing,
then you'll just have to address them when it comes to presenting results,
and then the project sponsors will finally have to start thinking about
what matters to them.  In the worst case, they just end up ignoring all
the test data as they can't decide on what really matters.

So, go and see what the project sponsors want and we can let you know
if there are any tried and tested approaches to measuring it!

Good Luck
Gilbert

then On Wed, 19 Mar 2003, Lynne Colgan wrote:

> I've been out of the usability/user centred design field for a few years,

> satisfaction (such as the SUMI questionnaire).  Any opinions on the
> usefulness of metrics, for instance to compare different versions of a
> product?

    --------------------------------------------------------------
     POSTINGS (in plain text): [email protected]
           SUBSCRIPTION CHANGES:  http://lists.syntagm.co.uk
            (or send email to mailto:[email protected])
    --------------------------------------------------------------