Re: Usability scorecard / data repository

I am surprised Jeff hasn't posted yet, but he has a very good website here:
http://measuringusability.com/ which was featured a few months back in ACM
Interactions. My team took the work he had done, adapted it a bit for our
needs, and created a scorecard that showed how a particular project's
usability changed over time.

The basics behind his research was to measure efficiency, effectiveness and
satisfaction (hello ISO). Borrowing heavily from his work, my team measured
efficiency by time to complete the task, effectiveness by whether a task is
completed and how many mistakes were made in the task, and satisfaction by
asking the user for a Likert scale rating. What this allowed us to do is
measure improvements in each task and overall change as the software was
refined. Later on, we began tracking number of recommendations, severity of
recommendations, and recommendations implemented as some teams were more
reluctant to make user interface changes than others. Overall, the focus was
on measuring within a project, as opposed to comparing between two projects
(it was rare that two projects had the same number of tasks of the same
complexity making comparisons a risky proposition).

All of this needs a strong communicator to correctly present what this
scorecard does and does not do. The emphasis was this was a secondary tool
that provided a high level glimpse into the usability of a software project.
His work was aimed at deriving a single score that measured a software
project's usability. We didn't test with enough users to give us enough data
worth rolling up into a single score.

Hope this helps,
Dana

    --------------------------------------------------------------
        Tip of the Day: Forward out-of-office replies to
                    mailto:[email protected]
     CHI-WEB: www.sigchi.org/web POSTINGS: mailto:[email protected]
              MODERATORS: mailto:[email protected]
       SUBSCRIPTION CHANGES & FAQ:  www.sigchi.org/web/faq.html
    --------------------------------------------------------------