Re: Who is in favour of empirical research approach?

Dear Miles,

You say: "I like José's distinction between empirical research and its  
subset, quantitative research. Not all empirical research is  
statistical -- qualitative methods can be empirical as well."

Jose's title was not clearly formulated in line with his intention:  
"Who is in favour of empirical research approach?" I doubt somebody in  
this group could not be in favour.

José's distinction came in response to my observation:

On Friday March 5, I wrote:
"Dear José,
Please look at all the collection of Information Design Journal, you  
will find lots of articles about empirical research, or do I not  
understand your point?
Also the book "Information design" Easterby and Zwaga is full of that,  
and there is more, of course. Attention: empirical research is not  
only quantitative research, nor statistical analysis."

I use empirical research all the time, including focus groups, user  
trips, etnographic observation, long interviews, and user testing. I  
also use statistical analysis, but far more seldom. I think this might  
be a common pattern for evidence-based work in visual communication  
design.

Cheers

Jorge


>
>
> -----Original Message-----
> From: [email protected]  
> [mailto:[email protected]] On  
> Behalf Of Caroline Jarrett
> Sent: Friday, March 05, 2010 4:03 PM
> To: 'Discussions about information design'
> Subject: Re: InfoD-Cafe: Who is in favour of empirical research approach?
>
> José explained:
>
>> I am developing a list of arguments in favour of (or against, if you like)
> empirical research.
>> But, I am focused on empirical research that involves statistical analyses
> only.
>
> Not sure if these examples are what you mean, but here goes anyway.
>
> 1. The Direct Marketing industry has focuses solely on A/B testing for ages.
> Send out a mail shot with half of the pieces done in style A and half in
> style B: whichever gets the best response wins.
>
> I learned about this approach from Rene Gnam's "Rene Gnam's Direct Mail
> Workshop" Prentice Hall (April 1990) - out of print, but cheap copies widely
> available second hand. (Actually, I may buy one: I lent my copy to someone
> who failed to return it).
>
> The results tend to be pieces that are remarkably ugly but effective.
>
> 2. More recently, a client of mine decided that he wanted to do A/B testing,
> which was easy for him to arrange, rather than anything else. We did end up
> with a better form. I wrote it up:
> Jarrett, C. and Minott, C. (2004) "Making a better web form" Proceedings of
> the Usability Professionals' Association Conference, Minneapolis, Minnesota,
> USA.
> Article is available as a .pdf from here:
> http://www.formsthatwork.com/Articles
>
> 3. Quite a few of us have been using statistical and numerical methods in
> usability, usually as adjunct or complementary techniques. For example, I do
> quite a lot of search log analysis for the Open University and we use other
> complementary numerical methods e.g. they do a lot of traffic analysis.
> However, like the thoughtful people they are, they regard these as part of
> an overall package that includes qual. methods as well. I extracted some
> examples of the ways that we use quantitative methods in this paper:
> Jarrett, C. et al " Using Measurements from Usability Testing, Search Log
> Analysis and Web Traffic Analysis to Inform Development of a Complex Web
> Site Used for Complex Tasks"
> http://portal.acm.org/citation.cfm?id=1601632.1601722&coll=GUIDE&dl=GUIDE&CF
> ID=80667064&CFTOKEN=87243617
>
> 4. Similarly, Tom Tullis at Fidelity has combined a variety of quantitative
> methods with qualitative methods, and also sometimes uses quant methods on
> their own. His approach is written up in:
>
> Tullis, T and Albert, W. "Measuring the User Experience: Collecting,
> Analyzing, and Presenting Usability Metrics" Morgan Kaufmann
>
> 5. There is a sort of fashion for numerical-only justification of design
> decisions going on in parts of the web design community. For example, just
> in the last few days, there's been a bit of discussion around the "madlibs"
> style form used by http://huffduffer.com/signup
> Luke Wroblewski justified his design decisions to use this style with
> conversion statistics:
> http://www.lukew.com/ff/entry.asp?1007
> but others have fought back with some further statistics:
> http://www.kalzumeus.com/2010/02/27/lesson-from-madlibs-signup-fad-do-your-o
> wn-tests/
> and scepticism about the way the stats have been generated:
> http://www.sitepoint.com/blogs/2010/03/03/madlib-style-forms-increase-conver
> sion-by-30-well-maybe/
>
>
> Best
> Caroline Jarrett
> www.formsthatwork.com
>
>
> ___________________________________________________________________
>
> Use the following address to post a message to all subscribers:
>  [email protected]
>
> To subscribe, unsubscribe or change your options, visit:
>  http://list.InformationDesign.org/mailman/listinfo/infodesign-cafe
>
> For all Information Design matters:
>  http://InformationDesign.org
>
> Problems? Write to:
>  [email protected]
> ___________________________________________________________________
>
> ___________________________________________________________________
>
> Use the following address to post a message to all subscribers:
>  [email protected]
>
> To subscribe, unsubscribe or change your options, visit:
>  http://list.InformationDesign.org/mailman/listinfo/infodesign-cafe
>
> For all Information Design matters:
>  http://InformationDesign.org
>
> Problems? Write to:
>  [email protected]
> ___________________________________________________________________
>
>


___________________________________________________________________

Use the following address to post a message to all subscribers: 
 [email protected]

To subscribe, unsubscribe or change your options, visit:
 http://list.InformationDesign.org/mailman/listinfo/infodesign-cafe

For all Information Design matters:
 http://InformationDesign.org

Problems? Write to:
 [email protected]
___________________________________________________________________