Re: Who is in favour of empirical research approach?

José explained:

> I am developing a list of arguments in favour of (or against, if you like)
empirical research.
> But, I am focused on empirical research that involves statistical analyses
only.

Not sure if these examples are what you mean, but here goes anyway.

1. The Direct Marketing industry has focuses solely on A/B testing for ages.
Send out a mail shot with half of the pieces done in style A and half in
style B: whichever gets the best response wins.

I learned about this approach from Rene Gnam's "Rene Gnam's Direct Mail
Workshop" Prentice Hall (April 1990) - out of print, but cheap copies widely
available second hand. (Actually, I may buy one: I lent my copy to someone
who failed to return it).

The results tend to be pieces that are remarkably ugly but effective.

2. More recently, a client of mine decided that he wanted to do A/B testing,
which was easy for him to arrange, rather than anything else. We did end up
with a better form. I wrote it up: 
Jarrett, C. and Minott, C. (2004) "Making a better web form" Proceedings of
the Usability Professionals' Association Conference, Minneapolis, Minnesota,
USA.
Article is available as a .pdf from here:
http://www.formsthatwork.com/Articles

3. Quite a few of us have been using statistical and numerical methods in
usability, usually as adjunct or complementary techniques. For example, I do
quite a lot of search log analysis for the Open University and we use other
complementary numerical methods e.g. they do a lot of traffic analysis.
However, like the thoughtful people they are, they regard these as part of
an overall package that includes qual. methods as well. I extracted some
examples of the ways that we use quantitative methods in this paper:
Jarrett, C. et al " Using Measurements from Usability Testing, Search Log
Analysis and Web Traffic Analysis to Inform Development of a Complex Web
Site Used for Complex Tasks" 
http://portal.acm.org/citation.cfm?id=1601632.1601722&coll=GUIDE&dl=GUIDE&CF
ID=80667064&CFTOKEN=87243617

4. Similarly, Tom Tullis at Fidelity has combined a variety of quantitative
methods with qualitative methods, and also sometimes uses quant methods on
their own. His approach is written up in:
  
Tullis, T and Albert, W. "Measuring the User Experience: Collecting,
Analyzing, and Presenting Usability Metrics" Morgan Kaufmann

5. There is a sort of fashion for numerical-only justification of design
decisions going on in parts of the web design community. For example, just
in the last few days, there's been a bit of discussion around the "madlibs"
style form used by http://huffduffer.com/signup
Luke Wroblewski justified his design decisions to use this style with
conversion statistics:
http://www.lukew.com/ff/entry.asp?1007
but others have fought back with some further statistics:
http://www.kalzumeus.com/2010/02/27/lesson-from-madlibs-signup-fad-do-your-o
wn-tests/
and scepticism about the way the stats have been generated:
http://www.sitepoint.com/blogs/2010/03/03/madlib-style-forms-increase-conver
sion-by-30-well-maybe/


Best
Caroline Jarrett
www.formsthatwork.com


___________________________________________________________________

Use the following address to post a message to all subscribers: 
 [email protected]

To subscribe, unsubscribe or change your options, visit:
 http://list.InformationDesign.org/mailman/listinfo/infodesign-cafe

For all Information Design matters:
 http://InformationDesign.org

Problems? Write to:
 [email protected]
___________________________________________________________________