20100922 - Elgin, Validating the User-CHAT (UET)

Validating the User-Centered Hybrid Assessment Tool (User-CHAT): A comparative usability evaluation by Elgin, Peter D., Ph.D., Kansas State University, 2007 , 168 pages; AAT 3259297 My Interest: (1) User-CHAT = UET. (2) Hybrid/combination of user testing, heuristic evaluation and cognitive walkthrough. (3) Effectiveness score. (4) Thoroughness, Validity. Action: To read specific parts of Dissertation in future. Therefore the present study had two principal objectives: (1) to validate a hybrid usability technique that identifies important and ignores inconsequential usability problems, and (2) to provide empirical performance data for several usability protocols on a variety of contemporary comparative metrics. The User-C entered H ybrid A ssessment T ool (User-CHAT) was developed to maximize efficient diagnosis of usability issues from a behaviorally-based perspective while minimizing time and resource limitations typically associated with usability assessment environments. Several characteristics of user-testing, the heuristic evaluation, and the cognitive walkthrough were combined to create the User-CHAT. Comments: Use-CHAT is UET. UEM used is hybrid of user testing, heuristic evaluation and cognitive walkthrough. This may be good reference/benchmark for my research. Two techniques generated comparison lists of usability problems---user-testing data and various inclusion criteria for usability problems identified by the User-CHAT, heuristic evaluation, and cognitive walkthrough. Overall the results demonstrated that the User-CHAT attained higher effectiveness scores than the heuristic evaluation and cognitive walkthrough, suggesting that it helped evaluators identify many usability problems that actually impact users, i.e., higher thoroughness, while attenuating time and effort on issues that were not important, i.e., higher validity. Comments: I want to read about how he score & analysed Effectiveness, Thoroughness and Validity. Furthermore, the User-CHAT had the greatest proportion of usability problems that were rated as serious, i.e., usability issues that hinder performance and compromise safety. Comment: How many evaluators? Who were the evaluators? How was usability problems rated – serious, what else? The User-CHAT's performance suggests that it is an appropriate usability technique to implement into the product development lifecycle. Chapter 1 -- Introduction Certification Environment Constraints Certification and Usability Usability Evaluation Methods User-testing Cognitive Walkthrough Heuristic Evaluation UEM Characteristics Incorporated into the User-CHAT Development of the User-CHAT Comparative Usability Evaluation UEM Performance Metrics * Thoroughness * Validity * Effectiveness * Reliability . * Time per Usability Problem Hypotheses. Chapter 2 -- Method Participants The System & Benchmark Tasks Procedure * User-Testing * User-CHAT. * Heuristic Evaluation * Cognitive Walkthrough Severity Ratings Summary of Procedure and Data Collection Data Synthesis.. Classifying Usability Problems into Heuristics Determining Unique or Shared Usability Problems Chapter 3 -- Results Time Spent per Usability Problem Usability Problem Partitions Detection Rates Thoroughness, Validity, and Effectiveness Reliability Severity Ratings Heuristics Classifications Generating a Comparison List without User-testing

Oct 20 - ProQuest dissertation search "Usability Evaluation"

Oct 20 - Keywords were "Usability Evaluation." Continued to search and download, as continuation from Oct 15.

Al-Nuaim, Hana Abdullah. Development and Validation of a Multimedia User Interface Usability Evaluation Tool in the context of Educational Web Sites. [Dissertation, Doctor of Science] George Washington University. Jan 31, 2000.

Andre, Terence S. Determining the Effectiveness of the Usability Problem Inspector: A Theory-Based Model and Tool for Finding Usability Problems. [Dissertation, PhD in Industrial & Systems Engineering] Virginia Polytechnic Institute and State University, Blacksburg, Virginia. April 3, 2000.

Baker, Kevin F. Heuristic Evaluation of Shared Workspace Groupware based on the Mechanics of Collaboration. [Thesis, M.Sc.] University of Calgary, Calgary, Alberta, Canada. May 2002.

Capra, Miranda G. Usability Problem Description and the Evaluator Effect in Usability Testing. [Dissertation, PhD in Industrial & Systems Engineering] Virginia Polytechnic Institute and State University, Blacksburg, Virginia. March 13, 2006.

Chang, Yaowen. A Theory-based Usability Study of the Mouseover Abstract Interface. [Dissertation, Doctor of Education] Columbia University. 2005.

Clapsaddle, Donna J. Measuring Usability: Categorically Modeling Successful Websites Using Established Metrics. [dissertation, Doctor of Professional Studies in Computing] Pace University. May 2004.

Dykstra, Dean Julian. A Comparison of Heuristic Evaluation and Usability Testing: The Efficacy of a Domain-specific Heuristic Checklist. [dissertation, PhD] Texas A&M University. Dec 1993.

Elgin, Peter D. VALIDATING THE USER-CENTERED HYBRID ASSESSMENT TOOL (USER-CHAT): A COMPARATIVE USABILITY EVALUATION. [Dissertation, PhD] Kansas State University, Manhattan, Kansas. 2007.

Faulkner, Laura Lynn. Structured Software Usability Evaluation: An Experiment in Evaluation Design. [Dissertation, PhD] University of Texas at Austin. May 2006.

Govindaraju, Majorkumar. Development of Generic Design Guidelines to Manufacture Usable Consumer Products. [Dissertation, PhD] University of Cincinnati. 1999.

Hebb, Christopher Louis. Website Usability Evaluation using Sequential Analysis. [dissertation, PhD] Indiana University, Bloomington. May 2005.

Ho, Janet Chingyun. Evaluation of A Virtual Campus: Bell University Labs. [Thesis, Master of Applied Science] University of Toronto. 2000.

Hu, Xiangqun. Development and Evaluation of a Web-based Architectural Design Tool. [Thesis, Master of Computer Science] Technical University of Nova Scotia, Halifax, Nova Scotia. 1997.

Ivory, Melody Yvette. An Empirical Foundation for Automated Web Interface Evaluation. [Dissertation, PhD in Computer Science] University of California at Berkeley. Fall 2001.

Jenkins, Lillie Ruth. DESIGNING SYSTEMS THAT MAKE SENSE: WHAT DESIGNERS SAY ABOUT THEIR COMMUNICATION WITH USERS DURING THE USABILITY TESTING CYCLE. [Dissertation, PhD] the Ohio State University. 2004.

Jobrack-McDaniel, Naomi Ruth. Remote versus Laboratory Usability Evaluations with Static versus Interactive Graphical User Interfaces. [Thesis, M.A.] San Jose State University. May 1999.