Re: Usability scorecard / data repository

>>> Usability scorecards cut both ways - it can make your work more
visible, but will also define metrics for you to be evaluated on. Is
number of studies per month or subjects run the best way to measure your
value? Probably not - those are just the easiest to measure, and that's
the trap. So lets back up: what problem are you trying to solve? What
alternatives to solving that problem are you considering besides
scorecards? (If scorecards are the only thing you're considering,
something's wrong. You wouldn't design a UI that way, right?)

I think the problem to solve has to do with having data to communicate
to upper management about use of UCD services across the organization.
Do we need more resources in certain areas?  Are certain parts of the
organization or certain products "weak" in user focus?  Upper management
is supportive of UCD and says a lot of the right things at a broad level
- perhaps more concrete data would help strengthen our case for
improvement in certain areas.  We do a fair amount of UCD education and
evangelism - effective, but it takes time.  I think some data like
"Product X ran 100 user tests this year, and a similar sized Product Y
ran just 5" bluntly makes a point.

Other areas in the company also have scorecards so that can be another
driver to have one.  Sure it puts your results out there and that can be
risky I suppose.  A bad result may also better illustrate a problem that
needs to be fixed.

I agree simple counts of number of user tests is not the best way to
measure value.  But it is a basic measure that probably should be
tracked.  What else should be measured?

--Chris 

    --------------------------------------------------------------
        Tip of the Day: Email mailto:[email protected]
               with any comments, questions or problems
     CHI-WEB: www.sigchi.org/web POSTINGS: mailto:[email protected]
              MODERATORS: mailto:[email protected]
       SUBSCRIPTION CHANGES & FAQ:  www.sigchi.org/web/faq.html
    --------------------------------------------------------------