20101013 - Yen, ...Usability Evaluation: Methods, Models, Measures

Health information technology usability evaluation: methods, models, and measures by Yen, Po-Yin, Ph.D., Columbia University, 2010 , 160 pages; AAT 3420882 My Interest: 1) Usability evaluation studies. 2) Usability evaluation scale (Health-ITUES). 3) Usability evaluation model (Health-ITUEM). 4) Validity. Action: To read the Dissertation in the future. Introduction Health information technology (IT) can offer important benefits to health care; however, technology-related factors are a major obstacle to health IT adoption. Research Goal Toward the goal of achieving a greater understanding of health IT usability and its measurement, the dissertation comprised three major analyses: 1) a methodological review of health IT usability evaluation studies to identify problems in existing studies; 2) exploratory factor analysis of the Health Information Technology Usability Evaluation Scale (Health-ITUES) which was developed as part of the dissertation research along with the underlying Health Information Technology Usability Evaluation Model (Health-ITUEM); and 3) confirmatory factor analysis and structural equation modeling to examine the construct validity and predictive validity of Health-ITUES. Methodology The health IT system that served as the focus of the analysis was a web-based communication system that supported nurse staffing and scheduling. The sample comprised 553 staff nurses in two healthcare organizations. In the usability methodological review, we identified problems in existing studies including lack of theoretical framework/model, inconsistent usability definition and evaluation methods, and lack of power analysis for sample size calculation. Results Discussion The exploratory factor analysis resulted in a 20-item Health-ITUES comprising four factors that demonstrated strong internal consistency reliability: quality of work life (QWL), 3 items, α=.94; perceived usefulness (PU), 9 items, α=.94; perceived ease of use (PEU), 5 items, α=.95; user control (UC), 3 items, α=.81. The confirmatory factor analysis showed that a general usability factor accounted for 78.1%, 93.4%, 51.0% and 39.9% of the explained variance in QWL, PU, PEU, and UC respectively. The structural equation modeling supported the predictive validity of Health-ITUES, explaining 64% of the variance in intention for system use. Contribution The results of the dissertation contribute to enhancing the methodological breadth and rigor of health IT usability evaluation studies. Chapter 1. Introduction 1 Background 2 Definitions of Usability 3 Aspects of Usability 7 Definition and Scope 8 Health IT Usability, Acceptance, and Adoption 10 Health IT Usability Specification and Evaluation 11 Problem Statement 12 Purpose 13 Study Aims and Research Questions 13 Significance of the Study 15 Chapter 2. Methodological Review of Health Information Technology Usability Specification and Evaluation Studies 16 Background 16 Usability Model 17 System Development Life Cycle 17 An Integrated Usability Specification and Evaluation Framework 18 Methods 23 Search Strategy 23 Inclusion/Exclusion Criteria: 23 Data Extraction and Management , 24 Results 27 Types of Health IT Evaluated 32 Summary of studies categorized in each stage 33 Stage 1: Specify Needs and Setting 35 Stage 2: System Component Development 37 Stage 3: Combination Components 38 Stage 4: Integrate Health IT into the Real Environment 40 Stage 5: Routine Use 41 Study Design and Data Analysis in Stages 4 and 5 41 Discussion 45 Methodological Problems in Existing Studies 45 Objective versus Subjective Measures 47 Environmental Factor Not Evaluated in the Early Stages 49 Inadequate Measure 49 Limitations 50 Conclusion 50 Chapter 3. Health IT Usability Evaluation Model and Scale Development 65 Background 65 Health IT Usability Evaluation Model 70 Definition of Concepts 74 Health IT Usability Evaluation Scale Development 76 The web-based communication system 76 Item selection, creation, and modification 78 Health IT Usability Evaluation Scale Psychometric Evaluation 82 Research Questions 82 Methods 82 Results 84 Discussion 92 Limitations 94 Conclusion 95 Chapter 4. Health-IT Usability Evaluation Scale Confirmatory Analyses 96 Background 96 Research Questions 96 Methods 97 Setting and Sample 97 Sample size 97 Data collection procedures 98 Data Analysis 98 Results 99 Descriptive analysis 99 Power Analysis 100 Construct validity 101 Predictive Validity 104 Discussion 106 Construct and Predictive Validity 106 Methodological issues in existing model testing studies 106 Limitations 107 Conclusions 108