Aug 29 - Masemola & de Villiers, Towards a Framework for Usability Testing of Interactive e-Learning Applications in Cognitive Domains...

Towards a Framework for Usability Testing of Interactive e-Learning Applications in Cognitive Domains, Illustrated by a Case Study.
S.S. (THABO) MASEMOLA AND M.R. (RUTH) DE VILLIERS. University of South Africa
S.S. Masemola, School of Computing, University of South Africa, P O Box 392, UNISA, 0003, South Africa; masemss@unisa.ac.za.
M.R. de Villiers, School of Computing, University of South Africa, P O Box 392, UNISA, 0003, South Africa; dvillmr@unisa.ac.za.
Proceedings of SAICSIT 2006, Pages 187 –197

Abstract
Testing has been conducted in a controlled usability laboratory on an interactive e-learning application that teaches mathematical skills in a cognitive domain. The study obtained performance measures and identified usability problems, but was focused primarily on using the testing technology to investigate distinguishing aspects of such applications, such as time usage patterns in domains where rapid completion is not necessarily a performance indicator. The paper addresses the issue of what, actually, is meant by ‘usability’ in learning environments. A pilot study identified obstacles and served to enhance the main study. Thinking-aloud on the part of participants provided useful data to support analysis of the performance measures, as can benchmarks and best case measures. Particular attention must be paid to ethical aspects. Features emerging from this study can contribute to a framework for usability testing and usage pattern analysis of interactive e-learning applications in cognitive domains.

What is Usability Testing?
Usability testing is a software evaluation technique that involves measuring the performance of typical end-users as they undertake a defined set of tasks on the system being investigated. It commenced in the early 1980s, as humanfactors professionals studied subjects using interfaces under real-world or controlled conditions and collected data on problems that arose (‘human factors’ is an early term for the human-computer interaction discipline). It has been shown to be an effective method that rapidly identifies problems and weaknesses, and is particularly used to improve the usability of products [Dumas, 2003; Dumas and Redish, 1999; Jeffries et al, 1991].
Since the early to mid-1990s, such testing has been empirically conducted in specialized controlled environments called usability laboratories, equipped with sophisticated monitoring and recording facilities for formal usability testing, supported by analytical software tools. It is an expensive technique. Participants, who are real end-users, interact with the product, performing specified representative tasks. Their actions can be rigorously monitored and recorded in various ways: by videotape − for subsequent re-viewing; event logging − down to keystroke level; and audio − to note verbalization and expressions.
The data, in both quantitative and qualitative forms, is analysed and changes can be recommended. Typical usability metrics include the time taken to complete a task, degree of completion, number of errors, time lost by errors, time to recover from an error, number of subjects who successfully completed a task, and so on [Avouris, 2001; Dix, Finlay, Abowd & Beale, 2004; ISO 9241, 1997; Preece, Rogers & Sharp, 2003; Wesson, 2002].

The primary targets of usability testing are the user interface and other interactive aspects. Such testing is used by academics for research and development, and also by usability practitioners in the corporate environment for rapid refinement of interfaces and analysis of system usability.

What is e-Learning?
Some definitions of e-learning equate it solely with use of the Internet in instruction and learning, but others [CEDEFOP, 2002; Wesson & Cowley, 2003] are broader, including multiple formats and methodologies such as the Internet and Web-based learning (WBL), multimedia CD-ROM, online instruction, educational software/courseware, and traditional computer-assisted learning (CAL). This approach suits the present study, which views e-learning as a
broad range of learning technologies encompassing various roles for technology, including interactive educational software, web-based learning, learning management systems, and learners using computers as tools [De Villiers, 2005].

Usability Testing
Dumas [2003] lists six defining characteristics of usability tests, while a seventh and eighth are obtained from Dumas and Redish [1999]:
1. The focus is usability.
2. Participants are end users or potential end users.
3. There is an artifact to evaluate, which may be a product design, a system or a prototype.
4. The participants think aloud as they progress through tasks.
5. Data is recorded and analysed.
6. The results are communicated to appropriate audiences (often a corporate client).
7. Testing should cover only a few features of the product, incorporated in selected tasks.
8. Each participant should spend approximately an hour doing the stated tasks.

Our methodology and test plan are based on general methodologies for formal usability testing [Pretorius, Calitz & Van Greunen, 2005; Rubin, 1994; Van Greunen & Wesson, 2002] but with some distinguishing features, such as the emphasis on participants thinking aloud and the use of a best case for initial benchmarking,...The broad methodology involves the following steps:
─ Set up objectives in line with research questions.
─ Determine the aspects to be measured and their metrics.
─ Formulate documents: Initial test plan, task list, information document for participants, checklist for administrator, and determine a means of investigating satisfaction.
─ Acquire representative participants.
─ Conduct a pilot test.
─ Refine the test plan, task list, and information document for the main usability test in the light of the pilot.
─ Conduct usability test.
─ Determine means of analysis and presentation that address the unique, as well as the usual, aspects.
─ Draw conclusions and make proposals for the way forward.


Conclusion

1. In addition to the identification of performance measures and problems, how can testing in a usability laboratory elicit valuable information about cognitive e-learning applications?
We propose that usability testing of e-learning applications should address both the interfaces and the learning content, because usability and functionality are closely related in e-learning. Performance metrics generated during learning activities can be used towards measuring the effectiveness aspect of usability, because success in the cognitive processing induced by learning functionality is fundamental to both usability and utility. It could be argued that conventional usability testing should be conducted on user interfaces only, to investigate users’ experiences with navigation features and the menus only. This could lead to improved interaction design, but it would be a paltry use of the capacity of the monitoring and analytical features in usability laboratories.

2. What activities/outputs yield meaningful information about interactive applications in such domains?
Innovative use of the usability lab technology led to usage analysis to identify usage patterns. Using data from thinking-aloud by subjects to clarify how they used their time, a clear distinction emerged between time spent navigating and time spent in cognitive activities. The act of configuring an environment or system to one’s own needs is called ‘incorporated subversion’ by Squires [1999]. In the present context, incorporated subversion of the usability testing technology led to added-value use of the laboratory and its software in novel and adaptable ways.

3. What notable features emerge from this study that can contribute specifically to a framework for usability testing of interactive e-learning in cognitive domains?
Generic frameworks and methodologies can be established, then customized for optimal usability testing of different kinds of e-learning applications and environments.
─ Attention to ethical aspects is of vital importance in this close-up recording of personal human activities.
─ A pilot study is essential to support the sensitive evolving plans and critical judgements that must be made.
─ This case study established the value of thinking out loud as a source of data, provided it is preceded by adequate preparation of participants.
─ The more fine-grained the tasks selected for testing, the better the data that is recorded.
─ Regarding the number of subjects, five are sufficient to identify usability problems, but are not enough to conduct serious analysis of learning and cognitive patterns. In this study the five participants generated valuable initial data on usage patterns and learning styles. Future in-depth research should be undertaken on low level, very finegrained tasks, accompanied by detailed analysis. With more data, realistic averages could be obtained to serve as benchmarks against which to compare future usage studies on small groups or single-subjects.
─ For the present, tentative benchmarks and best case measures were obtained. The best case provides a realistic optimum standard.

References that I may want to read further in future:
DE VILLIERS, M.R. 2004. Usability evaluation of an e-Learning tutorial: criteria, questions and a case study. In: G. Marsden, P. Kotzé, & A. Adesina-Ojo (Eds), Fulfilling the promise of ICT. Proceedings of SAICSIT 2004. ACM International Conference Proceedings Series.
DIX, A., FINLAY, J., ABOWD, G.D. and BEALE, R. 2004. Human-Computer Interaction. Pearson Education, Ltd, Harlow.
DUMAS, J.S. 2003. User-based evaluations. In: J.A. Jacko & A. Sears (Eds), The Human-Computer Interaction Handbook. Mahwah: Lawrence Erlbaum Associates.
DUMAS, J.S. and REDISH, J.C. 1999. A practical guide to usability testing. Exeter: Intellect.
FAULKNER, X. 2000. Usability Engineering. Houndsmills: Macmillan Press.
ISO 9241. 1997. Draft International Standard: Ergonomic requirements for office work with visual display terminals (VDT). Part 11: Guidance on Usability, ISO.
NIELSEN, J. 2000. Why you only need to test with five users. http://www.useit.com/alertbox/20000319.html . Accessed March 2006.
SQUIRES, D. and PREECE, J. 1999. Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers 11(5): 467–483.
VAN GREUNEN, D. and WESSON, J L. 2002. Formal usability testing of interactive educational software: A case study. World Computer Congress (WCC): Stream 9: Usability. Montreal, Canada, August 2002.
WESSON, J L. 2002. Usability evaluation of web-based learning: An essential ingredient for success, TelE-Learning 2002: 357- 363, Montreal, Canada, August 2002.
WESSON, J.L. and COWLEY, N. L. 2003. The challenge of measuring e-learning quality: Some ideas from HCI. IFIP TC3/WG3.6 Working Conference on Quality Education @ a Distance: 231-238, Geelong, Australia, February 2003.

Aug 29 - Ssemugabi & de Villiers, A Comparative Study of Two Usability Evaluation Methods Using a Web-Based E-Learning Application

A Comparative Study of Two Usability Evaluation Methods Using a Web-Based E-Learning Application.
Samuel Ssemugabi. School of Information Technology, Walter Sisulu University, Private Bag 1421, East London, 5200, South Africa. +27 43 7085407 ssemugab@wsu.ac.za
Ruth de Villiers. School of Computing, University of South Africa, P O Box 392, Unisa, 0003, South Africa +27 12 429 6559 dvillmr@unisa.ac.za
SAICSIT 2007, 2 - 3 October 2007, Fish River Sun, Sunshine Coast, South Africa.

ABSTRACT
Usability evaluation of e-learning applications is a maturing area, which addresses interfaces, usability and interaction from human-computer interaction (HCI) and pedagogy and learning
from education. The selection of usability evaluation methods (UEMs) to determine usability problems is influenced by time, cost, efficiency, effectiveness, and ease of application. Heuristic
evaluation (HE) involves evaluation by experts with expertise in the domain area and/or HCI.
This comparative evaluation study investigates the extent to which HE identifies usability problems in a web-based learning application and compares the results with those of survey evaluations among end-users (learners).
Severity rating was conducted on a consolidated set of usability problems and further comparison of findings was done on the major and minor problems. The results of HE correspond
closely with those of the survey. However the four expert evaluators identified more problems than the 61 learners and identified 91% of the learners’ problems, when major problems only were considered. HE by a competent and balanced set of experts showed itself to be an appropriate, efficient and effective UEM for e-learning applications.

Usability is a key issue in human-computer interaction (HCI) since it is the aspect that commonly refers to quality of the user interface [30].
The International Standards Organisation (ISO) defines usability as [16]: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context.

Usability evaluation is concerned with gathering information about the usability or potential usability of a system, in order to assess it or to improve its interface by identifying problems and
suggesting improvements [35]. Various usability evaluation methods (UEMs) exist, e.g. analytical, expert heuristic evaluation, survey, observational, and experimental methods [6,
13, 35].
To evaluate the usability of a system and to determine usability problems, it is important to select an appropriate UEM/s [10, 37], taking cognisance of efficiency, time, cost-effectiveness, ease of application, and expertise of evaluators [13, 30].

Evaluation of e-learning should address aspects of pedagogy and learning from educational domains as well as HCI factors such as the effectiveness of interfaces and the quality of usability and interaction.


Heuristic evaluation (HE) is a usability inspection technique originated by Nielsen [26, 27], in which a small set of expert evaluators, guided by a set of usability principles known as heuristics, determine whether a system conforms to these and identify specific usability problems in the system. It is the most widely used UEM for computer system interfaces. It is described as fast, inexpensive, and easy to perform, and can result in major improvements to user interfaces [4, 5, 19, 24].

Usability features are frequently not considered in development, sometimes because instructors and courseware developers are not trained to do so or because they lack the technological skills [41]. However, a main reason for the oversight is that usability evaluation can be difficult, time
consuming and expensive [20]. If, in addition to its attributes of low cost and relative simplicity, HE is shown to be effective, efficient, and sufficient to identify the problems that impede earners, it would qualify as an ideal method for evaluating webbased e-learning.

The main components of the research design
were:
1. Identification of the target application.
2. Generation of a set of criteria/heuristics suitable for usability evaluation of e-learning applications, in particular, WBL environments.
3. End-user evaluations by means of criterion-based learner surveys on the target application using questionnaires and a focus group interview.
4. An heuristic evaluation with a competent and complementary set of experts, using the synthesised set of criteria as heuristics, followed by severity rating on the consolidated set of problems.
5. Data analysis to answer the research question, investigating in particular how the usability problems identified by heuristic evaluation compare with with those identified by end users in the learner-surveys.
Evaluation criteria/heuristics (terms used interchangeably) for e-learning should address interfaces, usability and interaction from HCI, as well as pedagogy and learning from education.
A study was undertaken by the authors [38] to establish an appropriate set of 20 criteria within three categories for evaluating WBL applications.
This multi-faceted framework (see Table 1) supports Hosie’s and Schibeci’s concept [15] of ‘context-bound’/‘context-related’ evaluation. Nielsen’s [27] heuristics form the basis of Category 1, extended to customise them for educational purposes by using Squires’ and Preece’s [36] ‘learning with software’ heuristics. Other sources used in generating the criteria were, [1], [18], [19], [23], [34], [35] and [39].
Each criterion has a list of associated subcriteria or guidelines to help evaluators assess Info3Net. These are also shown in Table 1, but they can be customised to other contexts.

Category 1: General interface usability criteria (based on Nielsen’s heuristics, modified for e-learning context)
1 Visibility of system status
2 Match between the system and the real world i.e. match between designer model and user model
3 Learner control and freedom
4 Consistency and adherence to standards
5 Error prevention, in particular, prevention of peripheral usability-related errors [36]
6 Recognition rather than recall
7 Flexibility and efficiency of use
8 Aesthetics and minimalism in design
9 Recognition, diagnosis, and recovery from errors
10 Help and documentation
Category 2: Website-specific criteria for educational websites
11 Simplicity of site navigation, organisation and structure
12 Relevance of site content to the learner and the learning process
Category 3: Learner-centred instructional design, grounded in learning theory, aiming for effective learning
13 Clarity of goals, objectives and outcomes
14 Effectiveness of collaborative learning (where such is available)
15 Level of learner control
16 Support for personally significant approaches to learning
17 Cognitive error recognition, diagnosis and recovery
18 Feedback, guidance and assessment
19 Context meaningful to domain and learner
20 Learner motivation, creativity and active learning

SURVEY EVALUATIONS AMONG END-USERS (LEARNERS)
Query techniques, such as questionnaires and interviews aim to identify usability problems by asking users directly [9]. The questionnaire was designed using Gillham’s principles [12]. The first section elicited basic demographic information and details of the respondents’ experience. The main section was based on the 20 criteria of Section 5.1. For each criterion, straightforward, focused, single-issue statements, based on the sub-criteria, were generated to expand the meaning. Students rated Info3Net using the criteria, but a main aim was for them to use each criterion to name problems they had experienced in that regard, and write them in the space below.
Survey was done using 5-point Likert scale: Strongly agree, Agree, Maybe, Disagree, Strongly disagree.
Eighty registered students had used Info3Net during the semester, 61 of whom took part in the questionnaire survey, which was conducted on a single day in the usual laboratory setting, but in two separate groups. Sixty four problems were identified. To eliminate duplicates, a researcher meticulously considered them and combined those that were closely related. During consolidation, some were rephrased to correspond with terminology used by expert evaluators (7.4).
For further data, the questionnaire was followed by a focus group interview with eight students, as advocated by [35]. This clarified and elaborated problems identified by the questionnaire. More emerged, resulting in a final list of 55 problems.

HEURISTIC EVALUATION BY EXPERTS
Heuristic evaluation (HE) is an inspection technique whereby a set of experts evaluate whether a user interface conforms to defined usability principles, termed heuristics [9, 26, 33]. It is seldom possible for a single evaluator to identify all the usability problems in a system. Nielsen [27] recommends that a set ofthree to five evaluators be used to identify 65-75% of the usability problems. Our approach is based on Nielsen’s, using our custom-designed heuristics for web-based learning. The HE was supplemented by a severity rating process done by the experts to rank the final integrated set of problems ranked according to level of seriousness.

Identifying and Defining the Heuristics
The heuristics used were 15 of the 20 used in the end-user survey … and the results of the two evaluations are compared with regard to those 15. Each heuristic was divided into sub-criteria, as shown in Table 1.

Selection of Evaluators
Factors involved in selecting and inviting a balanced set of experts, are the number to use and their respective backgrounds.
Nielsen’s [27] cost-benefit analysis demonstrated optimal value with three or four evaluators. Despite this, the debate continues. [3] used eleven experts to assess usability of a university Web portal. Law and Hvannberg [22] reject the ‘magic five assumption’ and in the context of usability testing, used eleven participants to define 80% of the detectable usability problems. However, [19] argue that two to three evaluators who are experts both in the domain area and in HCI, so-called ‘double experts’, will point out the same number of usability problems as three to five ‘single experts’. Furthermore, in a heuristic evaluation of an educational multimedia application by Albion [1], four evaluators were selected, with expertise in user interface design, instructional/educational
design, and teaching.
The same approach was followed in this study. Four expert evaluators with expertise in user interface design, instructional/educational design, and teaching, were invited and agreed to participate. Two are lecturers in the subject-matter domain and are also HCI experts, familiar with HE; these two can be classified as ‘double experts’.

Briefing the Evaluators
Evaluators were briefed in advance about the HE process for the study, the domain of use of the target system, and the task scenarios to work through as advocated by [23] and [27]. In addition to the consent form (see 5.2) and a request to familiarise themselves with the heuristics before doing the actual evaluation, each evaluator was given a set of documents regarding the:
• Phases of the HE
• System and user profile
• Procedure.

Severity Rating
Severity rating, i.e assigning relative severities to individual problems, can be performed to determine each problem’s level of seriousness. It is usually estimated on a 3- or 5-point Likert
scale. The experts can either do ratings during their HEs or later, when all the problems have been aggregated.
The latter approach is advantageous [1, 23, 24] and is used in the present study. Table 5 shows the 5-point rating scale [32] used to assess the problems and assign severities. The final option indicates a non-problem.
The scale is similar to that used by Nielson [27] with an additional rating item, ‘Medium’ between the ‘Major’ and ‘Minor’ problem ratings.

Conclusion
The conclusion from these findings of a comparative evaluation study, is that the results of heuristic evaluation by experts correspond closely with those of survey evaluation among endusers (learners). In fact, the HE results are better than the survey results (see Table 6). They were produced by only four experts compared to 61 learners, and the experts were experiencing their first exposure to Info3Net, whereas the learners had used it for a semester.
The findings of this study indicate that heuristic evaluation, if conducted by a competent and complementary group of experts, is an appropriate, efficient, and highly effective usability evaluation method in the context of e-learning, as well as relatively easy to conduct and inexpensive. The researchers recommend that HE should, ideally, be supplemented with methods where users themselves identify usability or learning problems. This is in line with proposals that reliable evaluation can be achieved by systematically combining inspection with
user-based methods [2, 7]. In cases where only one approach has to be selected, the findings of this research can be used to propose heuristic evaluation as the optimal method.
A valuable secondary benefit of the study is that the problems identified in Info3Net can be addressed in future upgrades.
Another major contribution of this research is the generation of the framework of evaluation criteria, which is transferable to other e-learning contexts, where the sub-criteria can be customised to the particular environment or system.

Recommendations For Further Research
• Re-designing Info3Net in an action research approach, to solve some of the problems identified. The application could be re-evaluated to determine the impact of the changes.
• Applying the evaluation criteria generated in this study to evaluate other web-based e-learning applications.
• Adapting the framework of criteria to customise them for other contexts or other forms of e-learning.
• Determining which criteria are suitable for application by educators only and which by learners only.

References that I may want to read further in the future:
[1] Albion, P.R. 1999. Heuristic Evaluation of Multimedia: From Theory to Practice.
http://www.usq.edu.au/users/albion/papers/ascilite99.html
[2] Ardito, C., Costabile, M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T. & Rossano, V. 2006. An Approach to Usability Evaluation of E-Learning Applications. Universal Access in the Information Society, 4(3): 270-283.
[4] Belkhiter, N., Boulet, M., Baffoun, S. & Dupuis, C. 2003. Usability Inspection of the ECONOF System's User Interface Visualization Component. In: C. Ghaoui. (Ed.), Usability Evaluation of Online Learning Programs. Hershey, P.A.: Information Science Publishing.
[7] Costabile, M.F, De Marsico, M., Lanzilotti, R., Plantamura, V.L. & Roselli, T. 2005. On the Usability Evaluation of ELearning Applications. In: Proceedings of the 38th Hawaii International Conference on System Science: 1-10. Washington: IEEE Computer Society.
[8] De Villiers, R. 2006. Multi-Method Evaluations: Case Studies of an Interactive Tutorial and Practice System. In: Proceedings of InSITE 2006 Conference. Manchester, United Kingdom.
[12] Gillham, B. 2000. Developing a Questionnaire. London: Bill Gillham.
[14] Hartson, H.R., Andre, T.S. & Williges, R.C. 2003. Criteria for Evaluating Usability Evaluation Methods. International Journal of Human-Computer Interaction, 15(1): 145-181.
[15] Hosie, P. & Schibeci, R. 2001. Evaluating Courseware: A need for more context bound evaluations? Australian Educational Computing, 16(2):18-26.
[18] Jones, A., Scanlon, E., Tosunoglu, C., Morris, E., Ross, S., Butcher, P. & Greenberg, J. 1999. Contexts for Evaluating Educational Software. Interacting with Computers, 11(5): 499-516.
[19] Karoulis, A. & Pombortsis, A. 2003. Heuristic Evaluation of Web-Based ODL Programs. In: C. Ghaoui. (Ed.), Usability Evaluation of Online Learning Programs. Hershey, P.A.: Information Science Publishing.
[25] Masemola, S.S. & De Villiers, M.R. 2006. Towards a Framework for Usability Testing of Interactive E-Learning Applications in Cognitive Domains, Illustrated by a Case Study. In: J. Bishop & D. Kourie. Service-Oriented Software and Systems. Proceeding of SAICSIT 2006: 187-197. ACM International Conference Proceedings Series.
[26] Nielsen, J. 1992. Finding Usability Problems through Heuristic Evaluation. In: Proceedings of the SIGCHI Conference on Human factors in Computing Systems: 373-380. Monterey: ACM Press.
[27] Nielsen, J. 1994. Heuristic Evaluations. In: J. Nielsen & R.L. Mack. (Eds), Usability Inspection Methods. New York: John Wiley & Sons.
[28] Nielsen, J. & Molich, R. 1990. Heuristic Evaluation of User Interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems: Empowering People: 249-256. Seattle: ACM Press.
[30] Parlangeli, O., Marchingiani, E. & Bagnara, S. 1999. Multimedia in Distance Education: Effects of Usability on Learning. Interacting with Computers, 12(1): 37-49.
[31] Peng, L.K., Ramaiach, C.K, & Foo, S. 2004. Heuristic-Based User Interface Evaluation at Nanyang Technological University in Singapore. Program: Electronic Library and Information Systems, 38 (1): 42-59.
[32] Pierotti, D. 1996. Usability Techniques: Heuristic Evaluation Activities. http://www.stcsig.org/usability/topics/articles/heactivities.html
[36] Squires, D. & Preece, J. 1999. Predicting Quality in Educational Software: Evaluating for Learning, Usability and the Synergy Between them. Interacting with Computers, 11(5): 467-483.
[37] Ssemugabi, S. 2006. Usability Evaluation of a Web-Based ELearning Application: A Study of Two Evaluation Methods. MSc Dissertation, University of South Africa.
[38] Ssemugabi, S. & De Villiers, M.R. 2007. Usability and Learning: A Framework for Evaluation of Web-Based ELearning Applications. In: Proceedings of the ED-MEDIA 2007 - World Conference on Educational Multimedia, Hypermedia & Telecommunications: 906-913. Vancouver, Canada.
[39] Storey, M.A, Phillips, B., Maczewski, M. & Wang, M. 2002. Evaluating the Usability of Web-Based Learning Tools. Educational Technology & Society, 5(3):91-100.
[40] Van Greunen, D. And Wesson, J L. 2002. Formal usability testing of interactive educational software: A case study. World Computer Congress (WCC): Stream 9: Usability. Montreal, Canada, August 2002.
[41] Vrasidas, C. 2004. Issues of Pedagogy and Design in Elearning System. In: ACM Symposium on Online Learning: 911-915. Nicosia: ACM Press.
[42] Wesson, J.L. & Cowley, N. L. 2003. The Challenge of Measuring E-Learning Quality: Some Ideas from HCI. IFIP TC3/WG3.6 Working Conference on Quality Education @ a Distance: 231-238. Geelong, Australia.
[43] Zaharias, P. 2006. A Usability Evaluation Method for ELearning: Focus on Motivation to Learn. In: CHI '06 Extended Abstracts on Human Factors in Computing Systems: 1571-1576. Montreal: ACM Press.

My Comments: Ssemugabi and de Villiers had listed many references that may be useful to my research. Good hints for literature review. :)