Re: Usability Test Metrics

Lynne,
This is just my one point of view, but I do believe that it would be useful
to take standard usability metrics on the one fact that they can assist in
identifying specific parts of the new product that are not quite on the same
level as others.  For instance, you may find that one task that you gave the
person was 'suboptimal' (perhaps they couldn't even figure out how to do
it), but without taking some simple effectiveness and efficiency measures
you may actually miss out on that data - and data supporting an observation
is always very useful.  

I also believe that it is better to take excess data, then decide what makes
the strongest argument to management later (note: does the data not form the
argument for you?).  Perhaps you will only be interested in the SUMI data,
perhaps the effectiveness and efficiency measures actually provide some
additional justification for something.  I also feel that efficiency and
effectiveness measures are quite easy to take - have a stopwatch and time
each task, and count the number of errors or sub-goals achieved.  I don't
think much time or resources would be saved by removing them from the
analysis, but at the same time the data you lose may be 'unrecoverable',
especially since the greatest resource cost for usability testing following
the setup is the participants themselves (most times), and it would be
difficult to then go ahead and say 'I need more participants to get
additional data'. 

>From another perspective you said that efficiency and effectiveness data is
good when comparing to previous versions.  Perhaps when a new version comes
out of this product with supposed improvements then you already have data to
compare to, you won't need to retest the old system.

Todd

**
* Todd Bentley
*   SCHIL Usability Lab, Swinburne University of Technology
*   Hawthorn, Victoria, Australia
*   http://www.it.swin.edu.au/centres/schil/ul/
*
*   [email protected]
*   +61 (3) 9214 8860
** 

    --------------------------------------------------------------
     POSTINGS (in plain text): [email protected]
           SUBSCRIPTION CHANGES:  http://lists.syntagm.co.uk
            (or send email to mailto:[email protected])
    --------------------------------------------------------------