20100922 - Faulkner, Structured Software Usability Evaluation...

Structured software usability evaluation: An experiment in evaluation design by Faulkner, Laura Lynn, Ph.D., The University of Texas at Austin, 2006 , 192 pages; AAT 3225413 My Interest: (1) Detailed usability evaluation method and instrument (UET). (2) Experimental contemporary heuristics. (3) Comparison methods. (4) Structured Heuristic Evaluation Method (sHEM) by Masaaki Kurosu. Action: To read specific parts of the Dissertation in future. This research compared the effectiveness of a detailed usability evaluation method and instrument to a traditional overview method. Evaluating a product against a set of usability criteria is a technique known in the usability field as a "heuristic evaluation," during which a usability professional can identify problems before they reach the end user. Heuristic evaluations performed early in a product lifecycle can promote fixes in preproduction phases, rather than more costly testing and implementation phases. Methodology In the study, 63 usability professionals performed heuristic evaluations in a 2 x 2 research design to determine the relative effectiveness in revealing usability problems of using traditional versus experimental contemporary heuristics, each paired with an unstructured all-at-once evaluation method, or an experimental structured method. In the latter method, the evaluators were required to use limited sets of heuristics during a given session, with breaks between sessions. The work is an extension of research by Masaaki Kurosu, who developed the Structured Heuristic Evaluation Method (sHEM) for use in Japan, and tested it in a singlefactor design. In this present study the heuristic set and evaluation method variables were separated into a two-factor design; Kurosu's main effect of the contemporary/structured interaction was not replicated. Results Discussion Participants using traditional general heuristics found more usability problems than those using contemporary detailed heuristics. The study's strongest finding was that the structured approached rendered the participants more effective at identifying usability problems, even when individual participants found the approach "disconcerting" or "distracting." Results are congruent with the psychological phenomenon of retroactive interference: by interrupting the evaluation at intervals, participants were able to "forget" previous sections, freeing up working memory to find more usability problems in subsequent sections. Developing a New Usability Evaluation Tool * Designing the evaluation sheet. * Psychological foundations for the experimental evaluation sheet design. Testing the Tool Conclusion. CHAPTER TWO: METHODS Participants Materials and Test Procedure * The website designed and tested * Usability problems built into the test site. * Data collection sheets and experimental cells. * Test procedures. Demographic Data and Exit Survey Measures and Analysis Summary Comments: I like her style of having a Summary/Conclusion for each chapter.