Nov 4 - Rubin, Overview of Usability Testing

Chapter 2
Overview of Usability Testing


Goals of Testing

The overall goal of usability testing is to identify and rectify usability deficiencies existing in computer-based and electronic equipment and their accompanying support materials prior to release. The intent is to ensure the creation of products that:
* are easy to learn and to use.
* are satisfying to use.
* provide utility and functionality that are highly valued by the target population [49].

More specific goals or benefits of testing are:
* Creating a historical record of usabillity benchmarks for future releases.
* Minimizing the cost of service and hotline calls.
* Increasing sales and the probability of repeat sales.
* Acquiring a competitive edge since usability has become a market separator for products.
* Minimizing risks.

Limitations of Testing

Even the most rigorously conducted formal test cannot, with 100 percent certainty, ensure that a product will be usable when released.
Here are some reasons why:
* Testing is always an artificial situation.
* Test results do not prove that a product works.
* Participants are rarely fully representative of the target population.
* Testing is not always the best technique to use.

Basics of Test Methodology

Employing true experimental designs, these require that:
* A hypotheses must be formulated.
* Randomly chosen (using a very systematic method) participants must be assigned to experimental conditions.
* Tight controls must be employed.
* Control groups must be employed.
* The sample (of users) must be of sufficient size to measure statistical significance between groups [124].

Basic Elements of Usability Testing

1, Development of problem statements or test objectives rather than hypotheses.
2, Use of a representative sample of end users which may or may not be randomly chosen.
3. Representation of the actual work environment.
4. Observation of end users who either use or review a representation of the product. Controlled and sometimes extensive interrogation and probing of the participants by the test monitor.
5. Collection of quantitative and qualitative performance and preference measures.
6. Recommendation of improvements to the design of the product.

Four Types of Tests: An Overview

In this chapter I define and discuss three tests - exploratory, assessment, and validation tests - at a high level, according to the approximate point in the product life cycle at which each would be administered.
The fourth type of test, the comparison test, can be used as an integral part of any of the other three tests, and is not associated with any specific life cycle phase.


Source:
Jeffrey RUBIN. Handbook of Usability Testing: How To Plan, Design and Conduct Effective Tests. John Wiley & Sons, New York, 1994.