When to Use Which User Experience Research Methods
Summary: Modern day user experience research methods can now answer a wide range of questions. Knowing when to use each method can be understood by mapping them in 3 key dimensions and across typical product development phases.
The field of user experience, is blessed (or cursed) with a very wide range of research methods, ranging from tried-and-true methods such as lab-based usability studies to those that have been more recently developed, such as desirability studies (to measure aesthetic appeal).
You can't use the full set of methods on every project, but most design teams benefit from combining insights from multiple research methods.
The key question is what to do when. To better understand when to use which method, it is helpful to realize that they differ along 3 dimensions:
* Attitudinal vs. Behavioral
* Qualitative vs. Quantitative
* Context of Website or Product Use
* Attitudinal vs. Behavioral
* Qualitative vs. Quantitative
* Context of Website or Product Use
The Attitudinal vs. Behavioral Dimension
This distinction can be summed up by contrasting "what people say" with "what people do" (very often quite different).
The purpose of attitudinal research is usually to understand, measure, or inform change of people's stated beliefs, which is why attitudinal research is used heavily in marketing departments.
While most usability studies should rely more on behavior, methods that use self-reported information can still be quite useful. For example, card sorting provides you with insights about users' mental model of an information space, which can help you determine the best information architecture for your site. Surveys measure attitudes or collect self-reported data that can help track or discover important issues with your site. Focus groups tend to be less useful for usability purposes, for a variety of reasons.
On the other end of this dimension, methods that focus mostly on behavior usually seek to understand "what people do" with minimal interference from the method itself. A/B testing only changes the site's design, but attempts to hold all else constant, in order to see the effect of site design on behavior, while eyetracking seeks to understand how users visually interact with interface designs.
Between these two extremes lie the two most popular methods we use: usability studies and field studies. They utilize a mixture of self-reported and behavioral data, and can move toward either end of this dimension, though leaning toward the behavioral side is generally recommended.
The Qualitative vs. Quantitative Dimension
The basic distinction here is that, in qualitative studies, the data is usually being gathered directly, whereas in quantitative studies, the data is gathered indirectly, through an instrument, such as a survey or a web server log.
In field studies and usability studies, for example, the researcher directly observes how people use technology (or not) to meet their needs. This gives them the ability to ask questions, probe on behavior or possibly even adjust the study protocol to better meet its objectives. Analysis of the data is usually not mathematical.
By contrast, insights in quantitative methods are typically derived from mathematical analysis, since the instrument of data collection (e.g., survey tool or web-server log) captures such large amounts of data that are coded numerically.
Due to the nature of their differences, qualitative methods are much better suited for answering question about why or how to fix a problem, whereas quantitative methods do a much better job answering how many and how much type of questions.
The Context of Product Use Dimension
The final distinction has to do with how and whether participants in the study are using the website or product in question. This can be described by:
* Natural or near-natural use of the product
* Scripted use of the product
* Not using the product during the study
* A hybrid of the above .
* Natural or near-natural use of the product
* Scripted use of the product
* Not using the product during the study
* A hybrid of the above .
Phases of Product Development (the time dimension)
Another important distinction to consider when making a choice among research methodologies is the phase of product development and its associated objectives.
STRATEGIZE: In the beginning phase of the product development, you are typically considering new ideas and opportunities for the future. Research methods in this phase can vary greatly.
OPTIMIZE: Eventually, you will reach a "go/no-go" decision point, when you transition into a period when you are continually improving the design direction you have chosen. Research in this phase is mainly formative and helps you reduce the risk of execution.
ASSESS: At some point, the website or product will be available for use by enough users where you can begin measuring how well you are doing.
Strategize
Goal: Inspire, explore and choose new directions and opportunities
Approach: Qualitative and Quantitative
Typical methods: Ethnographic field studies, focus groups, diary studies, surveys, data mining or analytics
Typical methods: Ethnographic field studies, focus groups, diary studies, surveys, data mining or analytics
Optimize
Goal: Inform and optimize designs in order to reduce risk and improve usability
Approach: Mainly Qualitative (formative)
Typical methods: Cardsorting, field studies, participatory design, paper prototype and usability studies, desirability studies, customer emails
Assess
Goal: Measure product performance against itself or its competition
Approach: Mainly Quantitative (summative)
Typical methods: Usability benchmarking, online assessments, surveys, A/B testing
Source:
Jakob Nielsen's Alertbox, October 6, 2008:
When to Use Which User Experience Research Methods
When to Use Which User Experience Research Methods
When to Use Which User Experience Research Methods (Alertbox)
No comments:
Post a Comment