Sep 7 - Yeap, Who & When to do Heuristic Evaluation (part 9 of UIC for eLearn)

Nielsen's Heuristic Evaluation Methodology: Who To Do Heuristic Evaluation

Heuristic evaluation is conducted based on a set of guidelines or heuristics. Hence, it could be carried out by virtually anyone. Usability experts are generally more effective in identifying usability problems as compared to novice evaluators. Nielsen (1995a) stated that simple usability inspection could be carried out after a few hours of training.

“Heuristic evaluation was originally developed as a usability engineering method for evaluators who had some knowledge of usability principles but were not necessarily usability experts.” (Nielsen, 1992a)

Figure 2.3: Graph showing proportion of usability problems found against number of evaluators. Taken from: Nielsen (2000a), Why You Need To Test With 5 Users.

Nielsen’s recommendation is three to five evaluators. (Nielsen, 2000a; Nielsen, 2005b) Referring to Figure 2.3, three evaluators could detect about 65% of usability problems, and five evaluators could detect about 85%. Based on Pareto concept and 80/20 rule, five evaluators would be adequate to find at least 80% of the usability problems.

Number of evaluators for satisfactory identification of high proportion of usability problems is as below.
· Double specialists (usability specialists with experience in the specific type of user interface) – 2 to 3 persons (could detect 81 to 90% of usability problems).
· Usability specialists – 3 to 5 persons (could detect 74% to 87% of usability problems.
· Novice evaluators – 14 persons (could detect more than 75% of usability problems. By using 5 novice evaluators, 51% of the usability problems could be identified. (Nielsen, 1992a; Nielsen & Molich, 1990a)

Figure 2.4: Graph showing benefits-cost ratio against number of evaluators. Taken from: Nielsen (2005b), How to Conduct a Heuristic Evaluation.

“The optimal number of evaluators in this example is four, with benefits that are 62 times greater than the costs.” (Nielsen, 2005b) Referring to Figure 2.4, high benefit-cost ratio is achieved for heuristic evaluation involving three to six evaluators.


When To Do Heuristic Evaluation

Heuristic evaluation is relatively low cost to be conducted. It could be implemented at various stages of portal or website development cycle.

Heuristic evaluation could be conducted at the stages of:
a) Storyboard,
b) Design stage,
c) Prototype of user interface,
d) “Running version” of the prototype,
e) Partial completion of the e-Learning portal or website,
f) Completion of e-Learning portal,
g) Ongoing review of an existing running e-Learning portal.


_______
source:
Usability Inspection Criteria for e-Learning Portals.
TeckChong Yeap. MDP7515 PROJECT Dissertation. MULTIMEDIA UNIVERSITY, MALAYSIA, October 2008.