20101013 - Hollins, ...Instrument to assess Usability of courseware

The development of an instrument to assess the usability of courseware by Hollins, Louvenia D., Ed.D., Northern Illinois University, 2008 , 270 pages; AAT 3324334 My Interest: 1) UET = Instructional Design Usability Survey (IDUS). 2) Heuristic evaluation. 3) Empirical usability testing. 4) Walkthroughs. 5) Field observations. 6) Theoretical framework. 7) Three-phases approach. 8) ANOVA. 9) Validity & reliability of UET/Survey. Action: To read the Dissertation in future. Research Goal This study describes the development and initial validation of a survey designed to measure students' perceptions of the usability of instructional design of educational courseware. Motivation There is a lack of measuring instruments to conduct this kind of research. The present study attempts to take steps to provide educators and researchers with a useful tool to facilitate their studies of this aspect or courseware creation and use. Methodology A three-phase approach is taken in this study to develop the Instructional Design Usability Survey (IDUS) instrument. Expert evaluators from the field of educational technology, research and assessment were involved in Phase I to establish content and face validity. During Phase II, student focus groups were formed from the population of courseware users to establish content validity and clarity of the instrument. In Phase III, students examined the usability of the instrument, and reliability and validity tests were conducted. Comments: This dissertation ought to be read in detail. Because a UET is developed. Second reason is to benchmark how she validated the UET. Results Discussion Reliability and validity of scores from the IDUS were assessed through statistical procedures performed on the data collected from 206 student participants. Internal consistency reliability was assessed for each construct using Cronbach's alpha. The internal consistency for the six constructs ranged from .73 to .92. Confirmatory factor analysis (CFA) was conducted to establish construct validity by examining the overall fit of the models as well as the magnitude of the factor loadings for each proposed construct. Results of the study show the IDUS is reliable but does not yet show strong enough evidence of validity. 1. INTRODUCTION TO THE STUDY 1 Statement of the Problem 3 Purpose of the Study 4 Theoretical Framework 5 Research Questions 9 Definition of Terms 9 Significance of the Study 10 Organization of the Study 10 2. REVIEW OF RELATED LITERATURE 12 CAI and User ID Guidelines 12 HCI and UEM 26 Empirical Usability Testing 29 Heuristic Evaluation 29 Field Observations 30 Walkthroughs 30 Analytical Modeling Methods 31 User-Centered Design (USD) 32 Survey Development and Validity 33 Summary of the Literature Review 41 3. RESEARCH METHODOLOGY 43 Research Methodology 44 Three-Phase Approach Overview 44 Research Site and Student Target Population 44 Instrument Development 46 Phase I: Expert Evaluation 49 Expert Target Population 49 Data Collection 50 Data Analysis 52 Phase II: Student Focus Group 56 Data Collection 56 Findings of Phase II 60 Phase III: Student Pilot Test 61 Data Collection 62 Data Analysis 63 Chapter Page Missing Data 65 Summary 66 4. FINDINGS OF THE STUDY 68 Demographic Data Analysis 68 Research Question 1 73 Research Question 1A 73 Research Question IB 88 Research Question 2 97 Posttest Reading Levels ANOVA 98 Posttest Math Levels ANOVA 100 Summary 105