20101013 - Hollins, ...Instrument to assess Usability of courseware

The development of an instrument to assess the usability of courseware by Hollins, Louvenia D., Ed.D., Northern Illinois University, 2008 , 270 pages; AAT 3324334 My Interest: 1) UET = Instructional Design Usability Survey (IDUS). 2) Heuristic evaluation. 3) Empirical usability testing. 4) Walkthroughs. 5) Field observations. 6) Theoretical framework. 7) Three-phases approach. 8) ANOVA. 9) Validity & reliability of UET/Survey. Action: To read the Dissertation in future. Research Goal This study describes the development and initial validation of a survey designed to measure students' perceptions of the usability of instructional design of educational courseware. Motivation There is a lack of measuring instruments to conduct this kind of research. The present study attempts to take steps to provide educators and researchers with a useful tool to facilitate their studies of this aspect or courseware creation and use. Methodology A three-phase approach is taken in this study to develop the Instructional Design Usability Survey (IDUS) instrument. Expert evaluators from the field of educational technology, research and assessment were involved in Phase I to establish content and face validity. During Phase II, student focus groups were formed from the population of courseware users to establish content validity and clarity of the instrument. In Phase III, students examined the usability of the instrument, and reliability and validity tests were conducted. Comments: This dissertation ought to be read in detail. Because a UET is developed. Second reason is to benchmark how she validated the UET. Results Discussion Reliability and validity of scores from the IDUS were assessed through statistical procedures performed on the data collected from 206 student participants. Internal consistency reliability was assessed for each construct using Cronbach's alpha. The internal consistency for the six constructs ranged from .73 to .92. Confirmatory factor analysis (CFA) was conducted to establish construct validity by examining the overall fit of the models as well as the magnitude of the factor loadings for each proposed construct. Results of the study show the IDUS is reliable but does not yet show strong enough evidence of validity. 1. INTRODUCTION TO THE STUDY 1 Statement of the Problem 3 Purpose of the Study 4 Theoretical Framework 5 Research Questions 9 Definition of Terms 9 Significance of the Study 10 Organization of the Study 10 2. REVIEW OF RELATED LITERATURE 12 CAI and User ID Guidelines 12 HCI and UEM 26 Empirical Usability Testing 29 Heuristic Evaluation 29 Field Observations 30 Walkthroughs 30 Analytical Modeling Methods 31 User-Centered Design (USD) 32 Survey Development and Validity 33 Summary of the Literature Review 41 3. RESEARCH METHODOLOGY 43 Research Methodology 44 Three-Phase Approach Overview 44 Research Site and Student Target Population 44 Instrument Development 46 Phase I: Expert Evaluation 49 Expert Target Population 49 Data Collection 50 Data Analysis 52 Phase II: Student Focus Group 56 Data Collection 56 Findings of Phase II 60 Phase III: Student Pilot Test 61 Data Collection 62 Data Analysis 63 Chapter Page Missing Data 65 Summary 66 4. FINDINGS OF THE STUDY 68 Demographic Data Analysis 68 Research Question 1 73 Research Question 1A 73 Research Question IB 88 Research Question 2 97 Posttest Reading Levels ANOVA 98 Posttest Math Levels ANOVA 100 Summary 105

20100926 - Platt, ..Electronic Performance Support System...

The efficacy of an electronic performance support system as a training tool for online faculty by Platt, Jeffrey Lynn, Ph.D., Iowa State University, 2008 , 336 pages; AAT 3342280 My Interest: 1) Usability Evaluation form. 2) EPSS – Electronic Performance Support System. 3) Methodology of Studies. Action: To read specific parts of the Dissertation in future. Background An instructional website was developed as an electronic performance support system (EPSS) to determine whether faculty would use and learn from such a resource. This website summarized pedagogical information about online instruction that users could access 24 hours a day, seven days a week. Methodology In Phase I of the study, five college professors with experience teaching online courses evaluated the usability of the website and made suggestions on how it could be improved. Revisions were made to the website to enhance clarity and comprehension in preparation for Phase II of the study. In Phase II, 35 college instructors agreed to use the website to learn about pedagogical information related to teaching online courses. Only two people visited at least 75% of the web pages, limiting the conclusions that could be drawn. Conclusion Two conclusions, however, seem warranted. First, an EPSS is not the most effective way to deliver non-mandatory faculty education because the primary goal of an EPSS is to enhance performance in a business setting rather than to facilitate learning in the academic setting. Second, any type of faculty development program must be developed from the learner's perspective and include interaction between the participants. CHAPTER 3. MATERIALS AND METHODS 29 Overview 29 Design of Ped·a·GO!·gy 29 Phase I: Usability Evaluation of Ped·a·GO!·gy 35 Phase I Participants 35 Phase I Procedure 35 Phase II Use of Ped·a·GO!·gy as a Learning Tool 35 Phase II Participants 35 Initial Phase II Questionnaire 37 Section 1 of the Phase II Questionnaire 37 Demographic Questions 37 Software Experience 37 Section 2 of the Phase II Questionnaire 38 Innovation Adopter Status Scale 38 Online Learning Platform 39 Section 3 of the Phase II Questionnaire 39 Phase II Procedure 43 Phase II Measures 44 Phase II Performance Measures 44 Phase II Exit Evaluation 45 Innovation Attribute Scale 45 Additional Questions 46 Chapter 3 Summary 46 CHAPTER 4. RESULTS AND DISCUSSION 48 Results 48 Phase I: Usability Evaluation of Ped·a·GO!·gy 48 Phase II: Use of Ped·a·GO!·gy as a Learning Tool 53 Computer Experience of Participants 53 Innovation Adopter Classification of Participants 54 Website Activity 57 Measures of Learning 60 Exit Evaluation of Ped·a·GO!·gy 63 Follow-Up Questionnaire for Non-Completers 65 Discussion 66 Efficacy as an Electronic Performance Support System 66 Diffusion of Innovation as a Theoretical Foundation 70 Hypothetical Outcomes 73 For Whom Ped·a·GO!·gy is the Most Useful 73 Concept Check Quizzes and Case Studies 74 Exit Evaluation of Ped·a·GO!·gy 75 Chapter 4 Summary 76 APPENDIX A. Ped·a·GO!·gy's INSTRUCTIONAL MATERIAL 84 APPENDIX B. PHASE I USABILITY EVALUATION FORM 300 APPENDIX C. PHASE II INITIAL QUESTIONNAIRE 301 APPENDIX D. PHASE II EXIT EVALUATION 308 APPENDIX E. PHASE I EVALUATOR RESPONSES TO Ped·a·GO!·gy 311 APPENDIX F. FOLLOW-UP QUESTIONNAIRE FOR NON-COMPLETERS 320

20100926 - Harmons, Usability Study of Postconference Online Selfassessment...

A usability study of a postconference online selfassessment program for health care professionals by Harmons, Eric M., M.A., Texas Woman's University, 2005 , 49 pages; AAT 1428775 My Interest: 1) Usability analysis/study. 2) QUIS – Questionnaire User Interface Survey. 3) Phone interview. 4) Post-conference online assessment. 5) Blackboard Academic Suite. Action: To read the Dissertation in the future. Motivation Continuing education (CE) conferences are widely used by professionals for the enhancement of clinical proficiency. Professionals view these types of educational activities as a vital means to enhance their professional practice and maintain clinical competency. The current environment of continuing education conferences revolves around a single-phase (didactic only) approach to learning. Current literature supports the notion of a multi-phased approach to enhance the learning experience for the lifelong learner. Research Goal The purpose of this study was a pilot usability analysis of a post-conference online selfassessment, developed within a Blackboard Academic Suite(TM) (Blackboard) classroom. Methodology The Questionnaire User Interaction Survey (QUIS) and phone interviews were utilized to assess the user interaction satisfaction with completing the self-assessment program. Results Discussion The overall response of participants to the system was a mean of 7.86 (scale from 1--9) across all the categories of usability. Based on the results, the Blackboard system is viewed by healthcare professionals as an effective medium for delivering an online self-assessment. Participants felt the self-assessment provided an effective tool to reflect on the presented material and direct their learning. Comments: Softcopy Dissertation is scanned version; cannot copy and paste; difficult to blog about it.

20100925 - Young, Evaluation of dust control technologies...effectiveness, usability

Evaluation of dust control technologies for drywall finishing operations: Industry implementation trends, worker perceptions, effectiveness and usability by Young, Deborah Elspeth, Ph.D., Virginia Polytechnic Institute and State University, 2007 , 191 pages; AAT 3310218 My Interest: 1) Usability evaluation instrument. 2) How the 16 participants did usability evaluation. Action: Medium-low priority to read the specific parts of Dissertation in future. Problem Statement Drywall finishing operations have been associated with worker exposure to dust that contains known particulate respiratory health hazards, such as silica, talc, and mica. Despite the existence of engineering, work-practice, and personal protective equipment (PPE) control technologies for the mitigation of this hazard, worker exposures persist in the drywall finishing industry. Research Goal This research employed a macroergonomic framework to evaluate this problem and identify barriers to dust control technology adoption in the key subsystems: personnel, technological, and organizational. Methodology & Results – First Study In the first study, the organizational subsystem was evaluated through a telephone interview of 264 drywall finishing firm owners. This study found the most commonly used dust control technology was respiratory protection. Cost, usability, environmental factors, and productivity were barriers identified in preventing adoption of other technologies. Methodology & Results – Second Study In the second study, of the technological subsystem, 16 participants performed simulated drywall finishing tasks with each of four methods, in a laboratory setting. Dust particles were monitored and compared among the technologies used. Participants performed usability evaluations of the four tools. The ventilated sander produced less respirable-size class dust than did the other three tools. The block sander produced more dust than the other three tools. Usability evaluations revealed that the block sander was easiest to learn, easiest to use, and perceived to be the best overall, while the wet method and pole sander were considered to have poor usability in terms of ease of use and productivity. Usability problems associated with perceived comfort and ease of use were identified for the ventilated sander, but it was tied for "overall best" with the block sander. Methodology & Results – Third Study The third study, of drywall finishing worker perceptions, employed the Health Belief Model to assess barriers to technology adoption, risk, susceptibility, and benefits. Results showed that workers have a high perception of the risk associated with drywall dust, but a lower perception of individual susceptibility to disease as a result of occupational exposure. Barriers to the use of dust control technologies were identified as being associated with organizational and usability factors. Chapter 3: Evaluation of Drywall Sanding Technologies Chapter 4: Evaluation of Worker Perceptions Appendix B: Study 1 Survey Instrument Appendix C: In-depth Interview Guide Appendix D: Usability Evaluation Instrument – Individual Technology Appendix E: Usability Evaluation Instrument – Technologies Comparison

20100925 - Ryu, ..Usability Questionnaires for Electronic Mobile Products...

Development of usability questionnaires for electronic mobile products and decision making methods by Ryu, Young Sam, Ph.D., Virginia Polytechnic Institute and State University, 2005 , 206 pages; AAT 3177322 My Interest: 1) MPUQ – Mobile Phone Usability Questionnaire. 2) How to select usability dimensions & questionnaire items. 3) Redundancy & Relevancy analysis. 4) How to test/validate MPUQ. Action: To read the Dissertation in future. Problem Statement As the growth of rapid prototyping techniques shortens the development life cycle of software and electronic products, usability inquiry methods can play a more significant role during the development life cycle, diagnosing usability problems and providing metrics for making comparative decisions. A need has been realized for questionnaires tailored to the evaluation of electronic mobile products, wherein usability is dependent on both hardware and software as well as the emotional appeal and aesthetic integrity of the design. Research Goal This research followed a systematic approach to develop a new questionnaire tailored to measure the usability of electronic mobile products. The Mobile Phone Usability Questionnaire (MPUQ) developed throughout this series of studies evaluates the usability of mobile phones for the purpose of making decisions among competing variations in the end-user market, alternatives of prototypes during the development process, and evolving versions during an iterative design process. In addition, the questionnaire can serve as a tool for identifying diagnostic information to improve specific usability dimensions and related interface elements. Methodology Employing the refined MPUQ, decision making models were developed using Analytic Hierarchy Process ( AHP) and linear regression analysis. Next, a new group of representative mobile users was employed to develop a hierarchical model representing the usability dimensions incorporated in the questionnaire and to assign priorities to each node in the hierarchy. Employing the AHP and regression models, important usability dimensions and questionnaire items for mobile products were identified. Finally, a case study of comparative usability evaluations was performed to validate the MPUQ and models. A computerized support tool was developed to perform redundancy and relevancy analyses for the selection of appropriate questionnaire items. The weighted geometric mean was used to combine multiple numbers of matrices from pairwise comparison based on decision makers' consistency ratio values for AHP. The AHP and regression models provided important usability dimensions so that mobile device usability practitioners can simply focus on the interface elements related to the decisive usability dimensions in order to improve the usability of mobile products. The AHP model could predict the users' decision based on a descriptive model of purchasing the best product slightly but not significantly better than other evaluation methods. Mobile Phone Usability Questionnaire (MPUQ) Except for memorability, the MPUQ embraced the dimensions included in the other well-known usability definitions and almost all criteria covered by the existing usability questionnaires. In addition, MPUQ incorporated new criteria, such as pleasurability and specific tasks performance. 2.1. Subjective Usability Assessment 2.1.1. Definitions and Perspectives of Usability 2.1.2. Usability Measurements 2.1.3. Subjective Measurements of Usability 2.1.4. Usability Questionnaires 2.1.4.1. Definition of Questionnaire 2.1.4.2. Questionnaires and Usability Research 2.2. Mobile Device Usability 2.2.1. Definition of Electronic Mobile Products 2.2.2. Usability Concept for Mobile Device 2.2.3. Mobile Device Interfaces 2.2.4. Usability Testing for Mobile Device 3. PHASE I : DEVELOPMENT OF ITEMS AND CONTENT VALIDITY 3.1. Need for a New Scale 3.2. Study 1: Conceptualization and Development of Initial Items Pool 3.2.1. Conceptualization 3.2.2. Survey on Usability Dimensions and Criteria 3.2.2.1. Usability Dimensions by Early Studies 3.2.2.2. Usability Dimensions in Existing Usability Questionnaires 3.2.2.3. Usability Dimensions for Consumer Products 3.2.2.4. Items from a Usability Questionnaire for a Specific Product 3.2.3. Creation of an Items Pool 3.2.4. Choice of Format 3.3. Study 2: Subjective Usability Assessment Support Tool and Item Judgment 3.3.1. Method 3.3.1.1. Design 3.3.1.2. Equipment 3.3.1.3. Participants 3.3.1.4. Procedure 3.3.2. Result 3.3.2.1. Part 1. Redundancy Analysis 3.3.2.2. Part 2. Relevancy Analysis 3.3.3. Discussion 3.4. Outcome of Studies 1 and 2 4. PHASE II : REFINING QUESTIONNAIRE 4.1. Study 3: Questionnaire Item Analysis 4.1.1. Method 4.1.1.1. Design 4.1.1.2. Participants 4.1.1.3. Procedure 4.1.2. Results 4.1.2.1. User Information 4.1.2.2. Factor Analysis 4.1.2.3. Scale Reliability 4.1.2.4. Known-group Validity 4.1.3. Discussion 4.1.3.1. Eliminated Questionnaire Items 4.1.3.2. Normative Patterns 4.1.3.3. Limitations 4.2. Outcome of Study 3 6. PHASE IV : VALIDATION OF MODELS 6.1. Study 6: Comparative Evaluation with the Models 6.1.1. Method 6.1.1.1. Design 6.1.1.2. Equipment 6.1.1.3. Participants 6.1.1.4. Procedure 6.1.2. Results 6.1.2.1. Mean Rankings 6.1.2.2. Preference Data Format 6.1.2.3. Friedman Test for Minimalist 6.1.2.4. Friedman Test for Voice/Text Fanatics 6.1.2.5. Comparisons Among the Methods 6.1.2.6. Important Usability Dimensions 6.1.3. Discussion 6.1.3.1. Implication of Each Evaluation Method 6.1.3.2. PSSUQ and the MPUQ 6.1.3.3. Validity of MPUQ 6.1.3.4. Usability and Actual Purchase 6.1.3.5. Limitations 6.2. Outcome of Study 6