Determining the right number of users for User Research.
Most clients and projects require the design researchers to state a predetermined sample size of users. Researchers often find it hard to justify to clients and business stakeholders on the choice of their sample size of users.
Dealing with these competing priorities at each stage of product development
What matters more: killer UX that makes people want to use your product, or shipping the things people want quickly and staking down a huge share of the market? If the UX is bad, people won’t want to use it. On the other hand, if someone else gets it there first, people are happy to use what is available and help to improve it with feedback as it grows.
Well, it happened again – the same thing that happens every time any digital product is put through usability testing. We found out that the people designing the thing (people who know exactly what it’s supposed to do and how it’s supposed to work) are not the same as the people actually using the thing. And the people who are supposed to use the thing don’t get it. And because they don’t get it, they have three options for how they might respond.
Some people think that usability is very costly and complex and that user tests should be reserved for the rare web design project with a huge budget and a lavish time schedule. Not true. Elaborate usability tests are a waste of resources. The best results come from testing no more than 5 users and running as many small tests as you can afford.
In earlier research, Tom Landauer and I showed that the number of usability problems found in a usability test with n users is:
where N is the total number of usability problems in the design and L is the proportion of usability problems discovered while testing a single user. The typical value of L is 31%, averaged across a large number of projects we studied. Plotting the curve for L=31% gives the following result:
The most striking truth of the curve is that zero users give zero insights.
Konami recently sent us a copy of their new title, Castlevania: Lords of Shadow. After testing the game, it was clear that the game could be a case study to highlight the difference between usability and user-experience (UX).