Sep 23 - Nielsen, Acting on User Research (Alertbox)

Summary: User research offers a learning opportunity that can help you build an understanding of user behavior, but you must resolve discrepancies between research findings and your own beliefs.

Acting on User Research

The good news in user research is that we're building up a massive body of knowledge about user behavior in online systems.
Here's one way of quantifying the amount of current usability knowledge: We've published 3,326 pages of usability research reports. Those reports contain 1,217 general design guidelines and 1,961 screenshots, all of which have information about how specific designs helped or hindered real users.
Other researchers publish reports as well, and you may also have an internal usability group or have commissioned consultants to study issues of interest to you.
In total, a huge mass of user research. How should you deal with all these findings?

First Steps

User research is a reality check. It tells you what really happens when people use computers. You can speculate on what customers want, or you can find out.
Research offers an understanding of how users behave online, and is a solid foundation on which to build a design.
I still recommend that you user test your own design: any time you have a new idea, build a paper prototype and test it so that you don't waste money implementing ideas that don't work. But, if you start with design ideas that are based on the actual behavior of real human beings, you'll have considerably fewer usability problems than if you target a design at a hypothetical or idealized user.

It can be overwhelming at first to see a long list of new research findings. Try to process them in small bites.
For example, look at your homepage or a typical content page in light of the new findings. Print out a copy and circle each design element that might violate a design guideline or cause users problems.

You can also use research findings as a checklist: go through your own design one guideline at a time and see whether you comply. Whenever you're in violation of an established usability finding, you can dig deeper into that finding's underlying user research and learn more about it. With your new knowledge, you might decide to fix your design to make it compliant with users' typical behavior.

Handling Disagreements

There are two main reasons people disagree with a research study and its conclusions:
1. their own research shows something different, or
2. their personal opinions and preferences differ from the research recommendations.

If your research findings disagree with published results, you have two options.
First, it's possible that your study had a methodology flaw, so it's worth reviewing the study in light of the new results. There are numerous issues to consider to run a valid user test.
Second, it could be that you're dealing with a special case, and your findings actually are different. Such cases do exist, although exceptions are more rare than people would like to think.
If your own intuition disagrees with published findings, view it as a learning opportunity that can improve your future insights.
Design is a business decision. You should follow the data and do what generates the biggest profits for your company, not what wins design awards.

If you disagree strongly, you can always run a study of your design to determine whether you are one of those rare exceptions. General usability guidelines typically hold true in about 90% of cases. There are many special circumstances that make the remaining 10% sufficiently atypical such that the best solution will be something other than the normal recommendation.

If you run an online business, you're in the user experience business: all the value flows through a user interface.

Conclusion
It's essential to develop the expertise to interpret user research and an understanding of when to run usability studies. You still have to know how to deal with the reports and make the research findings relevant to your business.

Source:
Jakob Nielsen's Alertbox, November 8, 2004:
Acting on User Research
http://www.useit.com/alertbox/20041108.html
Acting on User Research (Jakob Nielsen's Alertbox)