I planned not to do any literature review today. But i felt uneasy not blogging. Today is a very happy day for me in Jakarta.
User Testing is Not Entertainment
Summary: Don't run your studies for the benefit of the people in the observation room. Test to discover the truth about the design, even when user tasks are boring to watch.
I'm seeing a disturbing trend in which user studies are geared to entertain rather than to reveal knowledge about the target designs. The people running such studies have the best of intentions: they want to show their clients well-known usability problems in a captivating way, and thereby gain support for design improvements.
It's intellectually dishonest to run a study in order to generate specific findings. It's very easy to bias the study so that the findings are not even real.
When you know what you want to find, it's easy to overlook other issues that might turn out to be the design's biggest problems.
In my experience, when clients ask you to study one aspect of their website (say, the navigation), the real problem is often something different (say, the content).
It's therefore essential to keep an open mind during user testing and expect the unexpected.
Typical Errors
Some common mistakes of entertainment-focused user testing include:
1 Screening for outgoing, articulate test participants so that you can get more elaborate quotes and comments. While it's certainly more engaging for observers to watch users who talk a mile a minute, it's also potentially biasing.
Out of the 233 guidelines for recruiting test participants, the number one guideline is to get a representative sample of customers.
2 Focusing the test on the site's sexy parts, while skipping parts that require users to read detailed descriptions of products and services. But such boring behaviors often close the sale, particularly for B2B sites where detailed product specs and whitepapers are a key part of the user experience.
3 Prompting users to talk in great detail about the problem your design team is currently debating.
Such feedback isn't reliable, however, if you've turned users' attention away from their tasks to grill them about multiple, alternative design ideas.
You have to watch users do things -- not listen to them talk about hypothetical designs they aren't using.
It's fine to test two or three alternative paper prototypes that you've quickly mocked up. It's not fine to ask users to speculate on "how would you like it if the site did this...," because what people say has no relation to what they'd do if they actually used such a user interface.
aiyah.....i could not concentrate. Just have to postone to tomorrow.
Usability As Showbiz
When you're selecting highlights from usability study videos, you shouldn't include a 10-minute clip of a user visiting 20 erroneous pages. Show the first one or two wrong clicks, then throw up a transition title that says "nine minutes later...," and conclude with the clip where the user says, "This is a horrible website. I can't find anything. I'm never coming back."
Two entertaining minutes beat ten boring ones in a usability presentation. Not only will your team members and executives pay more attention to your findings, they'll also be more likely to attend your next usability briefing.
To heighten drama, cut out the boring parts.
1 To present findings, it's okay to emphasize material that engages and motivates your audience.
For example, don't show a bunch of clips of your most introverted participants.
2 To discover findings, however, you can't bias your study and skip directly to the good parts for three reasons.
First, if you don't let users go through those 20 wrong page views, you won't learn as much about the flaws in navigation design.
Second, if you guide users past "boring" pages, you won't learn how users interpret those parts of the site.
Third, if you take users directly to the site's "interesting" parts, they'll arrive at those areas with a substantially different mental model and attitude than people who've had to struggle through page after irrelevant page to get there. Users are more inclined to quickly leave a section if they've struggled to reach it. In contrast, users easily guided there will have a more positive attitude and will dig harder; the section will (falsely) appear to be more successful as a result.
3 To document findings, your report should focus on the importance of the usability issues for the overall business, not on which issues generated the most colorful quotes.
Conclusion
In the long run, money talks, and you optimize ROI by emphasizing unbiased research and by watching behavior. Usability's role in a design project is to be the source of truth about what really works in the world, as opposed to the team's hopes for what might work. \
The less thoroughly you uncover the truth, the less your company will need you in the long run.
Source:
Jakob Nielsen's Alertbox, September 11, 2006:
User Testing is Not Entertainment
http://www.useit.com/alertbox/user-testing-showbiz.html
User Testing is Not Entertainment (Jakob Nielsen's Alertbox)
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment