Alertbox: How Many Test Users in a Usability Study?
Jakob Nielsen <[email protected]> Mon, 4 Jun 2012 12:30:39 -0400
| Newsgroups | gmane.comp.web.alertbox |
|---|---|
| Message-ID | <538d934a891681a5d850bb4e5f405d0f72c.20120604163025@mail184.us1.rsgsv.net> |
This is a multi-part message in MIME format
--_----------=_MCPart_2028315139
Content-Type: text/plain; charset="utf-8"; format="fixed"
Content-Transfer-Encoding: quoted-printable
The answer is 5, except when it's not. Most arguments for using more
test participants are wrong, but some tests should be bigger and
some smaller.
> http://www.useit.com/alertbox/number-of-test-users.html
Jakob Nielsen's Alertbox for June 4: How Many Test Users in a
Usability Study?
__________________________________________________________________
Usability Week 2012 conference
* Chicago: June 25-29
* Toronto: July 23-27
* Sydney: August 12-17
* San Francisco: September 23-28
* London: October 14-19
Full event program describing all full-day training courses.
> http://www.nngroup.com/events
__________________________________________________________________
Intranet Design Competition
The annual design competition to identify the world's 10 best intranets
is now open.
Call for Entries: Deadline June 27
> http://www.nngroup.com/reports/intranet/2013
__________________________________________________________________
Windows Phone User Testing
We recently conducted some usability research with Windows Phone in
Amsterdam. (Doing this research in Europe because Nokia is much more
popular there than in the U.S.) I was pleasantly surprised. The live
tiles worked well, and user reactions to the phone were positive.
On the other hand, task performance using actual websites was less
positive.
Our sample size was too small to conclusively state whether current
mobile sites work better or worse on Windows Phone than on other
popular phones. All I can say at this time is that you should consider
testing your mobile site with Windows Phone users if you expect this
platform to be important for you.
We will probably have more to say later, after the new version of
Windows Phone ships and we've tested it with more users in more
countries.
__________________________________________________________________
Russia: the New Web Power
The latest data on Internet growth in Europe shows that Russia now
has the largest number of users, surpassing earlier-adopters like
Germany and the U.K.
> http://www.economist.com/node/21555560
At the recent Usability Week conference in Amsterdam, Russia was the
country sending the most participants after the host country
(Netherlands) and its two neighboring countries (Germany and Belgium).
Also, across the full year of Usability Week events, we now have a
Russian company represented on the list of most-attending companies:
Odnoklassniki (a social network that's bigger than Facebook in Russia).
I believe this is the first time a company from Eastern Europe has made
this exclusive list of organizations that invest heavily in
usability.
In total, usability enthusiasts from 32 countries attended our
Amsterdam conference.
__________________________________________________________________
Drive Safely
The Network of Employers for Traffic Safety has a lot of good
resources that may save your life or that of your colleagues or
children, by encouraging them to reduce driver distraction.
> http://trafficsafety.org
__________________________________________________________________
Copyright =C2=A9 2012 Nielsen Norman Group, All rights
reserved.
You're receiving this newsletter because you subscribed at www.useit.co=
m/alertbox/subscribe.html
Our mailing address is:
Nielsen Norman Group
48105 Warm Springs Blvd.
Fremont, CA 94539-7498, USA
Is this email not displaying correctly?
[14]View it in your browser.
[15]unsubscribe from this list | [16]update subscription preferences
Links:
14. http://us4.campaign-archive2.com/?u=3D538d934a891681a5d850bb4e5=
&id=3Ddbca633046&e=3Df405d0f72c
15. http://useit.us4.list-manage.com/unsubscribe?u=3D538d934a891681=
a5d850bb4e5&id=3D7f29a2b335&e=3Df405d0f72c&c=3Ddbca633046
16. http://useit.us4.list-manage.com/profile?u=3D538d934a891681a5d8=
50bb4e5&id=3D7f29a2b335&e=3Df405d0f72c
--_----------=_MCPart_2028315139
Content-Type: text/html; charset="us-ascii"
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<!-- This is a simple example template that you can edit to create your own custom templates -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<!-- Facebook sharing information tags -->
<meta property="og:title" content="Alertbox: How Many Test Users in a Usability Study?">
<title>Alertbox: How Many Test Users in a Usability Study?</title>
<style type="text/css">
#outlook a{
padding:0;
}
body{
width:100% !important;
}
body{
-webkit-text-size-adjust:none;
}
@media only screen and (max-device-width: 720px){
.mobilenoshow{
display:none !important;
}
} @media only screen and (max-device-width: 720px){
table td,table th{
padding:2px !important;
}
} body{
margin:0;
padding:0;
}
img{
border:none;
font-size:14px;
font-weight:bold;
height:auto;
line-height:100%;
outline:none;
text-decoration:none;
text-transform:capitalize;
}
#backgroundTable{
height:100% !important;
margin:0;
padding:0;
width:100% !important;
}
body,.backgroundTable{
background-color:#FAFAFA;
}
li{
margin-top:0px;
margin-bottom:0px;
}
#templateContainer{
border:1px solid #DDDDDD;
}
h1,.h1{
color:#202020;
display:block;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:140%;
font-weight:bold;
line-height:100%;
margin-top:0px;
margin-bottom:10px;
text-align:left;
}
h2,.h2{
color:#202020;
display:block;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:130%;
font-weight:bold;
line-height:100%;
margin-bottom:10px;
text-align:left;
}
h3,.h3{
color:#202020;
display:block;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:120%;
font-weight:bold;
line-height:100%;
margin-bottom:10px;
text-align:left;
}
h4,.h4{
color:#505050;
display:block;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:110%;
font-weight:bold;
line-height:100%;
margin-bottom:10px;
text-align:left;
}
#templatePreheader{
background-color:#FAFAFA;
}
.preheaderContent div{
color:#101010;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:9px;
line-height:100%;
text-align:left;
}
.preheaderContent div a:link,.preheaderContent div a:visited{
color:#336699;
font-weight:normal;
text-decoration:underline;
}
.preheaderContent div img{
height:auto;
max-width:600px;
}
#templateHeader{
background-color:#FFFFFF;
border-bottom:0;
}
.headerContent{
color:#202020;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:34px;
font-weight:bold;
line-height:100%;
padding:0;
text-align:center;
vertical-align:middle;
}
.headerContent a:link,.headerContent a:visited{
color:#336699;
font-weight:normal;
text-decoration:underline;
}
#headerImage{
height:auto;
max-width:600px !important;
}
#templateContainer,.bodyContent{
background-color:#FDFDFD;
}
.bodyContent div{
color:#101010;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
line-height:120%;
text-align:left;
}
.bodyContent div a:link,.bodyContent div a:visited{
color:#0033aa;
font-weight:normal;
text-decoration:underline;
}
.bodyContent img{
display:inline;
margin-bottom:10px;
}
#templateFooter{
background-color:#FDFDFD;
border-top:0;
}
.footerContent div{
color:#707070;
font-family:Verdana,Calibri,Helvetica,Arial,sans-serif;
font-size:12px;
line-height:125%;
text-align:left;
}
.footerContent div a:link,.footerContent div a:visited{
color:#0033aa;
font-weight:normal;
text-decoration:underline;
}
.footerContent img{
display:inline;
}
#social{
background-color:#FAFAFA;
border:1px solid #F5F5F5;
}
#social div{
text-align:center;
}
#utility{
background-color:#FDFDFD;
border-top:1px solid #F5F5F5;
}
#utility div{
text-align:center;
}
#monkeyRewards img{
max-width:160px;
}
</style></head>
<body leftmargin="0" marginwidth="0" topmargin="0" marginheight="0" offset="0">
<center>
<table border="0" cellpadding="0" cellspacing="0" height="100%" width="100%" id="backgroundTable">
<tr>
<td align="center" valign="top">
<table border="0" cellpadding="0" cellspacing="0" width="100%" id="templateContainer">
<tr class="mobilenoshow">
<td align="center" valign="top">
<!-- // Begin Template Header \\ -->
<table border="0" cellpadding="0" cellspacing="0" width="100%" id="templateHeader">
<tr>
<td class="headerContent">
<!-- // Begin Module: Standard Header Image \\ -->
<div style="text-align: left;"><a href="http://useit.us4.list-manage1.com/track/click?u=538d934a891681a5d850bb4e5&id=9c48e600b6&e=f405d0f72c" style="color: #336699;text-decoration: underline;font-weight: normal;"><img src="http://www.nngroup.com/images/nnglogo.gif" alt="" border="0" style="margin: 0;padding: 0;border: 0;height: auto;line-height: 100%;outline: none;text-decoration: none;" width="90" height="50"></a></div>
<!-- // End Module: Standard Header Image \\ -->
</td>
</tr>
</table>
<!-- // End Template Header \\ -->
</td>
</tr>
<tr>
<td align="center" valign="top">
<!-- // Begin Template Body \\ -->
<table border="0" cellpadding="10" cellspacing="0" width="100%" id="templateBody">
<tr>
<td valign="top" class="bodyContent">
<!-- // Begin Module: Standard Content \\ -->
<table border="0" cellpadding="10" cellspacing="0" width="100%">
<tr>
<td valign="top">
<div><div style="text-align: left">
<blockquote style="padding-bottom: 4px; line-height: 130%; background-color: #ffffdd; margin: 0px; padding-left: 4px; padding-right: 4px; color: #000000; padding-top: 4px">
<p style="margin-top: 0px; margin-bottom: 2px">
<a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=a0b1ae4787&e=f405d0f72c" style="color: #505050; text-decoration: none"><span style="color: #505050">Summary:</span><br>
The answer is 5, except when it's not. Most arguments for using more test participants are wrong, but some tests should be bigger and some smaller.</a></p>
<p style="line-height: 140%; margin-top: 4px; margin-bottom: 0px; font-size: 120%">
<a href="http://useit.us4.list-manage1.com/track/click?u=538d934a891681a5d850bb4e5&id=1b5c1e60d3&e=f405d0f72c" style="color: #505050; text-decoration: none">Jakob Nielsen's Alertbox for June 4: <span style="color: #0033aa; text-decoration: underline"><strong>How Many Test Users in a Usability Study? </strong></span></a></p>
</blockquote>
<hr>
<h3>
Usability Week 2012 conference</h3>
<ul>
<li>
<a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=47d367deeb&e=f405d0f72c">Chicago</a>: June 25-29</li>
<li>
<a href="http://useit.us4.list-manage1.com/track/click?u=538d934a891681a5d850bb4e5&id=ffed9a3248&e=f405d0f72c">Toronto</a>: July 23-27</li>
<li>
<a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=4bd39754db&e=f405d0f72c">Sydney</a>: August 12-17</li>
<li>
<a href="http://useit.us4.list-manage2.com/track/click?u=538d934a891681a5d850bb4e5&id=bbc789b6fe&e=f405d0f72c">San Francisco</a>: September 23-28</li>
<li>
<a href="http://useit.us4.list-manage2.com/track/click?u=538d934a891681a5d850bb4e5&id=7621dff795&e=f405d0f72c">London</a>: October 14-19</li>
</ul>
<p>
<a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=114d93f19b&e=f405d0f72c">Full event program</a> describing all full-day training courses.</p>
<hr>
<h3>
Intranet Design Competition</h3>
<p>
The annual design competition to identify the <strong>world's 10 best intranets</strong> is now open.</p>
<p>
<a href="http://useit.us4.list-manage2.com/track/click?u=538d934a891681a5d850bb4e5&id=24da844f3a&e=f405d0f72c">Call for Entries</a>: Deadline June 27</p>
<hr>
<h3>
Windows Phone User Testing</h3>
<p>
We recently conducted some usability research with Windows Phone in Amsterdam. (Doing this research in Europe because Nokia is much more popular there than in the U.S.) I was pleasantly surprised. The live tiles worked well, and user reactions to the phone were positive.</p>
<p>
On the other hand, task performance using actual websites was less positive.</p>
<p>
Our sample size was too small to conclusively state whether current mobile sites work better or worse on Windows Phone than on other popular phones. All I can say at this time is that you should consider testing your mobile site with Windows Phone users if you expect this platform to be important for you.</p>
<p>
We will probably have more to say later, after the new version of Windows Phone ships and we've tested it with more users in more countries.</p>
<p>
<em>See also:</em> Nice (but looong) article on the <a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=b7d5298762&e=f405d0f72c">interaction design concepts in Windows 8</a>.</p>
<hr>
<h3>
Russia: the New Web Power</h3>
<p>
The latest data on <a href="http://useit.us4.list-manage2.com/track/click?u=538d934a891681a5d850bb4e5&id=bf88b79c2c&e=f405d0f72c">Internet growth in Europe</a> shows that Russia now has the largest number of users, surpassing earlier-adopters like Germany and the U.K.</p>
<p>
At the recent Usability Week conference in Amsterdam, Russia was the country sending the most participants after the host country (Netherlands) and its two neighboring countries (Germany and Belgium).</p>
<p>
Also, across the full year of Usability Week events, we now have a Russian company represented on the list of most-attending companies: Odnoklassniki (a social network that's bigger than Facebook in Russia). I believe this is the first time a company from Eastern Europe has made this exclusive <a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=a6fdc83ce6&e=f405d0f72c">list of organizations that invest heavily in usability</a>.</p>
<p>
In total, usability enthusiasts from <strong>32 countries </strong>attended our Amsterdam conference.</p>
<hr>
<h3>
Drive Safely</h3>
<p>
The <a href="http://useit.us4.list-manage.com/track/click?u=538d934a891681a5d850bb4e5&id=e70bf95f1b&e=f405d0f72c">Network of Employers for Traffic Safety</a> has a lot of good resources that may save your life or that of your colleagues or children, by encouraging them to reduce driver distraction.</p>
<hr>
</div>
</div>
</td>
</tr>
</table>
<!-- // End Module: Standard Content \\ -->
</td>
</tr>
</table>
<!-- // End Template Body \\ -->
</td>
</tr>
<tr>
<td align="center" valign="top">
<!-- // Begin Template Footer \\ -->
<table border="0" cellpadding="10" cellspacing="0" width="100%" id="templateFooter">
<tr>
<td valign="top" class="footerContent">
<!-- // Begin Module: Standard Footer \\ -->
<table border="0" cellpadding="10" cellspacing="0" width="100%">
<tr>
<td valign="top" width="100%">
<div>
<em>Copyright © 2012 Nielsen Norman Group, All rights reserved.</em>
<br>
<!--
-->
You're receiving this newsletter because you subscribed at www.useit.com/alertbox/subscribe.html
<br>
<strong>Our mailing address is:</strong>
<br>
<div class="vcard"><span class="org fn">Nielsen Norman Group</span><div class="adr"><div class="street-address">48105 Warm Springs Blvd.</div><span class="locality">Fremont</span>, <span class="region">CA</span> <span class="postal-code">94539-7498, USA</span></div><br><a href="http://useit.us4.list-manage.com/vcard?u=538d934a891681a5d850bb4e5&id=7f29a2b335" class="hcard-download">Add us to your address book</a></div>
<br>
<!--
-->
</div>
</td>
<td valign="top">
<div>
<!--
-->Is this email not displaying correctly?<br>
<a href="http://us4.campaign-archive2.com/?u=538d934a891681a5d850bb4e5&id=dbca633046&e=f405d0f72c" target="_blank">View it in your browser</a>.<!--
-->
</div>
</td>
</tr>
<tr>
<td colspan="2" valign="middle" id="utility">
<div>
<a href="http://useit.us4.list-manage.com/unsubscribe?u=538d934a891681a5d850bb4e5&id=7f29a2b335&e=f405d0f72c&c=dbca633046">unsubscribe from this list</a> | <a href="http://useit.us4.list-manage.com/profile?u=538d934a891681a5d850bb4e5&id=7f29a2b335&e=f405d0f72c">update subscription preferences</a>
</div>
</td>
</tr>
</table>
<!-- // End Module: Standard Footer \\ -->
</td>
</tr>
</table>
<!-- // End Template Footer \\ -->
</td>
</tr>
</table>
<br>
</td>
</tr>
</table>
</center>
<img src="http://useit.us4.list-manage.com/track/open.php?u=538d934a891681a5d850bb4e5&id=dbca633046&e=f405d0f72c" height="1" width="1"></body>
</html>
--_----------=_MCPart_2028315139--