Sep 18 - Nielsen, Discount Usability: 20 years (Alertbox)

Discount Usability: 20 Years.

Summary:
Simple user testing with 5 participants, paper prototyping, and heuristic evaluation offer a cheap, fast, and early focus on usability, as well as many rounds of iterative design.

History of Discount Usability Evaluation
In September 1989, I presented a paper entitled "Usability Engineering at a Discount" at the 3rd International Conference on Human-Computer Interaction in Boston. In so doing, I officially launched the discount usability movement, though I'd been working out the ideas during the 2 years prior to that talk.
The idea, like many, was born of necessity. As a university professor in the 1980s I had a much smaller budget than, say, the IBM User Interface Institute at the T.J. Watson Research Center, where I'd worked before rejoining academia. Clearly, I couldn't do the same kind of high-budget usability projects that the fancy labs did, so I turned this to my advantage and started developing a methodology for cheap usability.

3 main components of discount usability (as per 1989 paper):
1 Simplified user testing, which includes a handful of participants, a focus on qualitative studies, and use of the thinking-aloud method. Although thinking aloud had been around for years before I turned it into a discount method, the idea that testing 5 users was "good enough" went against human factors orthodoxy at the time.
2 Narrowed-down prototypes — usually paper prototypes — that support a single path through the user interface. It's much faster to design paper prototypes than something that embodies the full user experience. You can thus test very early and iterate through many rounds of design.
3 Heuristic evaluation in which you evaluate user interface designs by inspecting them relative to established usability guidelines.

The gold standard then (20 years ago) was elaborate (and expensive) studies with quantitative metrics. Even now, I appreciate the place of this older approach, and we sometimes run benchmark studies for both our independent research and for bigger clients who want to track metrics despite the expense.

Discount usability often gives better results than deluxe usability because its methods drive an emphasis on early and rapid iteration with frequent usability input.

Although my 20 years of campaigning for discount usability have certainly not been in vain, I can't yet declare a win:
- Most companies still waste their money testing more than 5 users per usability testing round. My 1989 paper actually advocated testing 3 users, which usually gives the highest ROI. I've since backtracked from this radical position and now advocate 5 users as a compromise that will
work in most organizations, even if they're not as fast-moving as I envisioned designers to be in 1989.
- People still pay far more attention to questionable quantitative studies than they do to simpler qualitative studies that have much greater validity.
- Most design teams still don't believe in paper prototyping, preferring instead to spend considerable time creating elaborate design representations before they start collecting user feedback.
- Many people still reject the value of usability guidelines and of heuristic evaluation, even though it's become the second-most popular usability method.

My comments: usability testing is quantitative studies whereas heuristic evaluation is qualitative studies.

Bad user testing beats no user testing.

source:
Jakob Nielsen's Alertbox, September 14, 2009:
Discount Usability: 20 Years
http://www.useit.com/alertbox/discount-usability.html
Discount Usability: 20 Years (Jakob Nielsen's Alertbox)