Re: Usability scorecard / data repository

Usability scorecards cut both ways - it can make your work more visible, but
will also define metrics for you to be evaluated on. Is number of studies
per month or subjects run the best way to measure your value? Probably not -
those are just the easiest to measure, and that's the trap.

So lets back up: what problem are you trying to solve? What alternatives to
solving that problem are you considering besides scorecards? (If scorecards
are the only thing you're considering, something's wrong. You wouldn't
design a UI that way, right?)

> We also have a desire to create a more robust knowledge base of our
usability data

I've never seen a usability database that was worth the effort to create it.
Reports are rarely read,  certainly not old ones, and no programmer or
manager is going to spend time querying a database when easier alternatives
(asking usability engineers or making things up) exist.

The best thing I've seen to solve this problem is to write summary reports
every quarter. If you've done 10 studies on site navigation, at the end of
the quarter someone writes a high level summary of what was learned across
all those studies, focused on recommendations (not data) and lessons learned
that will be of greatest value to non-usability engineers (and pick summary
topics with this in mind).  Individual studies are often myopic and without
summary reports the real lessons learned never get documented.

Do a short presentation, open to all, on what was learned - those
sessions/summaries will get read and often have more impact than the studies
themselves.

-Scott


----- Original Message -----
From: "Chris Jasek" <[email protected]>
To: <[email protected]>
Sent: Tuesday, March 06, 2007 12:44 PM
Subject: Usability scorecard / data repository


> What are your experiences?  What is the best way to get the most out of
> tracking activity and archiving usability data without too much effort?

    --------------------------------------------------------------
    Tip of the Day: Suspend your subscription if using auto replies
     CHI-WEB: www.sigchi.org/web POSTINGS: mailto:[email protected]
              MODERATORS: mailto:[email protected]
       SUBSCRIPTION CHANGES & FAQ:  www.sigchi.org/web/faq.html
    --------------------------------------------------------------