20100922 - Dykstra, Domain-specific Heuristic Checklist

A comparison of heuristic evaluation and usability testing: The efficacy of a domainspecific heuristic checklist by Dykstra, Dean Julian, Ph.D., Texas A&M University, 1993 , 228 pages; AAT 9410761 My Interest: (1) Calendar Checklist = UET = domain-specific heuristics checklist. (2) Comparison methodology of the 3 usability evaluation methods. (3) Results comparison – software developers vs usability professionals. Action: To read this Dissertation in future. Motivation The need for cost-effective usability evaluation has led to the development of alternative usability methods. Reducing the number of participants in a usability test or using heuristic evaluation are the two approaches most often associated with discount methods. Research Objective In this dissertation, an alternative heuristic method was developed based on the concept of a double specialist (knowledge of usability techniques and of a software product domain). Double specialists were previously shown to provide heuristic evaluation superior to that of regular usability professionals, primarily because of their knowledge of usability problems typically found within the given domain. Similar knowledge, based on competitive usability testing, was used to devise a domain-specific heuristic checklist. Context The product domain chosen for the research was online calendars. The calendar checklist was compared to general heuristics and to streamlined usability testing with two usability test subjects. Comments: 3 evaluation types were probably: (1) calendar checklist (domain-specific heuristics checklist), (2) general heuristics, (3) streamlined usability testing. Methodology Fifteen software developers and fifteen usability professionals were each randomly assigned to one of the three evaluation methods. They were then provided with a previously unseen software calendar and were asked to find as many usability problems as possible. Results Discussion Participants using the calendar checklist found more total problems, and a significantly higher percentage of user-oriented problems and severe problems than did those using the general heuristic method, suggesting that domain-specific heuristics do provide an improved heuristic approach. Streamlined usability testing, however, was found to be the most effective discount usability method. Usability professionals and software developers did not differ in their ability to find user-oriented or severe problems. Conclusion It was concluded that when time and resources permit, usability testing is the method of choice. When user testing is not feasible, however, domain-specific heuristics may provide results approaching the effectiveness of the double specialist. Comment: I enjoy reading this Abstract; written beautifully and systematically.