Usability Guidelines for Designing Mobile Learning Portals.
Daniel Su Kuen Seong. The University of Nottingham, Malaysia. Jalan Broga, 43500 Semenyih, Selangor Darul Ehsan, Malaysia. +603-89248138 daniel.su@nottingham.edu.my
Mobility 06, Oct. 25–27, 2006, Bangkok, Thailand.
The 3rd International Conference on Mobile Technology, Applications and Systems — Mobility 2006.
Abstract
Mobile learning presumes the use of mobile Internet technology to facilitate the learning process. The growth and rapid evolution of the wireless technology have created new opportunities for anytime and anywhere learning paradigm. As a result, numerous mobile learning portals have been developed to gain the advantages of it. Nonetheless, there is little research and exploration has been initiated in proposing usability guidelines in designing mobile learning portals to achieve efficiency, effectiveness and satisfaction of learning. Thus, this paper seeks to present usability guidelines by grounding the user interface on usability theoretical framework, possible constraints, and unique properties of mobile computing. Three categories of usability have been formulated: user analysis, interaction and interface design. Ten golden usability guidelines have been suggested which aims for designing highly efficacious, user friendly and usable mobile interface to support dynamicity of mobile and handheld devices. Moreover, Mobile Learning Course Manager (MLCM) portal has been developed to demonstrate and exemplify the usability guidelines proposed.
This integration of technologies has altered considerably the instructional strategies in our educational institutions and changed the way teachers teach and students learn [2]. ... Some noticeable examples with the use of technologies in educational context are multimedia learning, World Wide Web or web/Internet-based learning, e-Learning, and in recent years, the mobile learning.
Mobile learning has been perceived by many educationalists to offer flexibility in learning and
present a multitude yet unique educational advantages [5]. The rapid evolving of the wireless communication and demanding for the low-cost of mobile devices potentially direct many researchers and educationalists to move from web-based and e-Learning to mobile learning which promising easy and convenient ways of learning.
One distinct feature of mobile learning over e-Learning is mobility [6]. Hence, researchers and scholars are becoming enthusiastically in coining the term ‘mobile learning or m-learning’, such as ‘mobile learning as the point at which mobile computing and e-Learning intersect to produce an anytime, anywhere learning experience [7].
According to Nyiri [8], m-learning is fundamentally e-Learning delivered through mobile computational devices such as Palms, Personal Digital Assistants (PDA), Pocket PCs, smart phone, digital cell phones, and any other handheld devices. The use of mobile devices with the wireless network technology flourishes mobile learners to get convenience, expediency and immediacy of mobile learning in appropriate time and accessing the appropriate learning contents [7].
Additionally, mobile learning is the next generation of e-Learning and important instrument for lifelong learning [49].
The term usability is defined as by The International Organisation for Standardisation (ISO) ISO 9241-11 [16] as ‘the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.’
According to Nielsen [17], usability means the measure of the quality the users’ experience when interacting interface.
...usability is not a surface gloss which applied at the last minutes or before the releases of the system or product; but it is deeply affected by every stage of the analysis, design, and development [18].
Usable systems are easy to learn (learnability), efficient to use (efficiency), easy to remember (memorability), not error-prone (errors), and satisfactory in use (subjective satisfaction) [17].
The ultimate goal of usability is meeting the needs of to users’ satisfaction [17].
..advantages of usability encompass increased productivity, enhanced quality of work, improved user satisfaction, and reductions in support and training costs [19]. The reduction in costs has attracted many project managers and interface designers to employ the theory of usability when designing the interfaces as reported in [20, 21].
Mobile learning appears in any facets of possible variations in context such as locations, environment, conditions, noisy or quiet, weather and so on. Hence, the dynamicity has triggered
additional efforts and investigation in order to design highly usable mobile learning portals to cater for different types of users in the global economy.
Furthermore, the limitation of screen size, the presentation of mobile contents, and adaptation of the information to the sensitivity of context and devices influence the efficiency and effectiveness when learning via the mobile devices.
Hayhoe [23] emphasises that the design criterion of this ubiquitous perspective need to be examined thoroughly and focused as to eliminate boredom and disorientation which involves wide range of users in having variety types of handheld devices.
It is perceptible that little research and exploration has been initiated to propose usability
guidelines in designing mobile learning portals. Moreover, several extensive experiments conducted from 2002 to 2005 discover substantial amount of severe usability problems
pertinent to user behaviour in mobile navigation on mobile Internet portals’ design [24]. Therefore, this has set the directions for this paper to explore the usable guidelines when
designing mobile learning portals by grounding the user interface on usability theoretical framework, possible constraints, and unique properties of mobile computing.
Category Usability Guideline
User Analysis U1
Interaction U2, U3, U4, U5 and U6
Interface design U7, U8, U9 and U10
The usability guidelines are divided into three categories: user analysis, interaction, and interface design.
Guidelines mean ‘rules or principles for action, encapsulating some combination of practitioner determined best practices in a domain and research-based insights into factors relevant in that
domain [25].’
The usability guidelines framework is grounded on the usability theoretical framework discussed in [17], constraints proposed by [22], and understanding of a number of unique properties that are intrinsic to mobile computing as highlighted in [52, 53, 54].
3.1.1 User Analysis
U1: The user/learner
3.1.2 Interaction
U2: Human-mobile interaction
U3: Map between mobile learning portals and the real world
U4: Help users recognise, diagnose, and recover from errors
U5: Visibility of the status
U6: Minimise human cognitive load
3.1.3 Mobile Learning Interface Design
U7: The small screen display
U8: Do not overuse
U9: Navigation
U10: Consistency.
My Comments: Majority of Seong's USABILITY GUIDELINES were taken from Nielsen's TEN HEURISTIC or TEN HEURISTIC PRINCIPLES. Nielsen's 10 were among my "36 USABILITY INSPECTION CRITERIA" for e-Learning Portals. Seong had also the smart thing; take the Usabilty Criteria from a very well-known source. Most researchers would use some, most or all of Nielsen's 10 usability criteria.
Principally MLCM portal’s user interfaces can be perceived usable as the mobile learners or users are considered the most important entity in our design. Much effort has been devoted in analysing and evaluating the mobile learners’ need and caters for beginner, fast acting for a more expert learner, provides efficacious support to the learners’ working needs and is pleasant to use.
CONCLUSIONS
Technological changes and evolution significantly amplifies the demands on the quality and usable user interface and offer the potential to further enhance the task, structure and functionality of mobile devices. The rapidly increased number of mobile learning portals has urged the usability issues to be more transparent when designing the learning portals. This paper contributes ten usability guidelines which envelop user analysis, interaction and user interfaces categories. These usability guidelines empower the effort in creating highly usable and impressive user interface which promote the users’ satisfaction and interactivity while learning via the mobile and handheld devices. In addition, the proposed usability guidelines intend to set as benchmark for interface designers to further evaluate the mobile user interfaces to conform to the attributes of usability when designing usable mobile learning portals. Additionally, MLCM has demonstrated and exemplified the use of usability guidelines proposed which aims to increase the users’ satisfaction especially dealing with the diversify and universal mobile learners who comes from different background, culture, race, educational level, learning cognitive and styles. The simplicity design and ease of use, consistency in navigation, structural metaphoric, and standardised iconic design have promote mobile users’ cognitive and learning styles in accessing and comprehending MLCM to maximise their learning experiences and increase their learning curve indirectly.
Future Directions
we suggest the following for future extensions:
• Examines the usability guidelines by conducting usabilitytesting/evaluation to measure and quantify the theory of usability such as learnability, memorability, efficiency, effectiveness, and subjective satisfaction. The expected results and outcomes are used to prioritise the importance and plot the ranking of it.
• Further evaluation and experiment will be conducted to assess and determine whether the usability guidelines proposed are appropriate and met the basis of formal instructional design strategies, and the mobile curriculum development.
• Enhances the learning portals by extracting the lecture notes and other relevant digital documents, and transmit them to the respective mobile learners.
My Comments: In my Masters research, I measured the relative importance of all the 36 USABILITY INSPECTION CRITERIA (Seong called them Usability Guideline) and ranked them according to the rating of importance. I hope to repeat this for a larger set (36 plus More) of usability criteria for mobile learning.
References that I may want to read further in future:
[16] ISO/IEC, 9241-11 Ergonomic Requirements for Office Work with Visual Display Terminals (VDT)s – Part 11 Guidance on Usability. 1998: ISO/IEC 9241-11: 1998 (E).
[18] Lee, K. B., and Grice, R. A. Developing a New Usability Testing Method for Mobile Devices. In Proceedings of the 23rd International Performance, Computing, and Communications Conference (IPCCC’04). 14-17 April 2004, Phoenix, Arizona, 115-127.
[19] ISO/IEC, 13407. Human-Centred Design Processes for Interactive Systems. 1999: ISO/IEC 13407: 1999(E).
[22] Mayhew, D. J. The Usability Engineering Lifecycle: A Practitioner’s Handbooks for User Interface Design. Morgan Kaufman Publisher, 1999.
[23] Hayhoe, G. F. From desktop to palmtop: Creating usable online documents for wireless and handheld devices. In Proceedings of the IEEE International Professional Communication Conference. Santa Fe, New Mexico, USA, 24-27 October 2001.
[24] Kaikkonen, A. Usability problems in today’s mobile Internet portals. In Proceedings of the 2nd IEE International Conference on Mobile Technology, Applications and Systems. 15-17 November 2005, Guangzhou, Chine, 459-464.
[25] Vavoula, G. N., Lefrere, P., O’Malley, C., Sharples, M., and Taylor, J. Producing Guidelines for learning, teaching and tutoring in a mobile environment. In Proceedings of the 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE’ 04). 23-25 March 2004, Jhongli, Taiwan, ROC, 173-176.
[30] Su, D. K. S. A comparative evaluation and correlation between learning styles and academic achievement on e-Learning. In Kwan, R., and Fong, J. (Eds.), Proceedings of Web-based Learning: Technology and Pedagogy at The 4th International Conference on Web-based Learning (ICWL’05). 31 July-3 August 2005, Hong Kong, China, World Scientific Publishing, Singapore, 193-202
[39] Kaikkonen, A., and Laarni, J. Designing for small display screens. In Proceedings of the 2nd Nordic Conference on Human-Computer Interaction. Aarhus, Denmark, New York: ACM Press, 19-23 October, 2002, 227- 230.
[42] Su, D. K. S., and Chan, F. C. Navigational patterns on usable mobile news portals. Journal of Internet Technology, 7(3), April 2006, forthcoming.
Showing posts with label Sharples. Show all posts
Showing posts with label Sharples. Show all posts
Friday, August 28, 2009
Thursday, August 27, 2009
Aug 28 - Vavoula & Sharples, Challenges in Evaluating Mobile Learning
Challenges in Evaluating Mobile Learning.
Giasemi N. Vavoula. University of Leicester, 105 Princess Road East, Leicester, LE1 7LG, UK. gv18@leicester.ac.uk Tel: +44 (0) 116 2523966
Mike Sharples. LSRI, University of Nottingham, Exchange Building, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, UK. mike.sharples@nottingham.ac.uk Tel: +44 (0) 115 9513716
Published in Proceedings of Mlearn2008.
ABSTRACT
We propose six challenges in evaluating mobile learning: capturing and analysing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing in/formality. A three-level framework for evaluating mobile learning is presented, comprising a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organisational contexts. The paper concludes with a discussion of how the framework meets the evaluation challenges and with suggestions for further extensions.
We now appreciate mobile learning not just as learning that is facilitated by mobile technology, but also as the processes of coming to know through conversations and explorations across multiple contexts amongst people and personal interactive technologies (Sharples et al. 2007b). Such evolving conceptions introduce numerous challenges to all aspects of mobile learning research, including evaluation. As the field matures, our frameworks and tools need to respond to these challenges.
CHALLENGE 1: CAPTURING LEARNING CONTEXT AND LEARNING ACROSS CONTEXTS
A major task for educational evaluation is to capture and analyse learning in context.
To appreciate the challenge, let us compare mobile learning contexts with traditional classroom contexts from the researcher’s perspective.
In order to establish and document the learning context, the researcher needs to know:
the location of learning and the layout of the space (where); the social setting (who, with whom, from whom); the learning objectives and outcomes (why and what); the learning method(s) and activities (how); and the learning tools (how).
CHALLENGE 2: HAS ANYONE LEARNED ANYTHING?
A second challenge that faces mobile learning evaluation is the assessment of learning processes and outcomes.
In traditional learning settings such as the classroom there are well-established and accepted methods for the assessment of learning activities, such as essay writing, multiple choice tests, open-book exams, and unseen examinations.
Distinctions have been made between formative assessment (aiming to provide students with feedback regarding their progress) and summative assessment (aiming to judge and sum up the students’ achievements) (Scriven 1967)...
CHALLENGE 3: AN ETHICAL QUESTION
Mobile learning can involve the use of mobile technology, which may also be personal technology. Tapping into a person’s mobile phone to find out how they have been using it to learn might mean invading that person’s privacy.
Although research ethics frameworks are implemented by most research institutions and organisations, mobile learning research raises profound ethical issues.
CHALLENGE 4: LET’S NOT FORGET THE TECHNOLOGY
Evaluations of mobile learning often reference inherent limitations of mobile devices, such as their small screens, short battery lives, intermittent connectivity, and associated human factors, all of which affect their usability. As the focus of research shifts from the mobility of the technology to the mobility of the learner, additional issues arise as learners move across multiple devices, possibly over short time periods in multiple locations. Assessing the usability of the mobile technology and the effectiveness of its integration with the mobile learning practice remains a high priority for evaluation.
CHALLENGE 5: SEEING THE BIGGER PICTURE
Oliver and Harvey (2002) suggest four different kinds of impact of educational technologies in Higher Education (HE): impact on students’ learning, impact on individual academics’ practice, impact on institution, and national impact.
In the same context of HE Price and Oliver (2007) identify three types of impact studies: anticipatory, ongoing and achieved.
Anticipatory studies relate to pre-intervention intentions, opinions and attitudes; ongoing studies focus on analysing processes of integration; and achieved studies are summative studies of technology no longer ‘novel’.
Riley (2007) extends this impact framework by distinguishing between minor modifications and culturally significant changes in practice, and suggesting that different kinds of change will emerge over different timescales.
Although not exclusively linked with HE contexts, mobile learning evaluation has similar questions to answer regarding impact.
CHALLENGE 6: FORMAL OR INFORMAL?
Mobile learning is often defined in terms of the technology that mediates the learning experience: if the technology is mobile, so is the learning. Mobility, however, is not an exclusive property of the technology, it also resides in the lifestyle of the learner, who in the course of everyday life moves from one context to another, switching locations, social groups, technologies and topics; and learning often takes place inconspicuously or is crammed in the short gaps between these transitions. Although this view of learning is inclusive of formal education contexts, it is particularly pertinent to everyday, informal learning.
Nevertheless, characterising a learning experience as formal or informal can be complicated. For example, is the learning of pupils visiting a museum (largely considered an informal learning setting) with their school (an irrefutably formal learning setting) a case of formal or informal learning?
REQUIREMENTS FOR MOBILE LEARNING EVALUATION
The challenges discussed in the previous sections indicate that mobile learning evaluation needs to:
• Capture and analyse learning in context with consideration of learner privacy
• Assess the usability of the technology and how it affects the learning experience
• Look beyond measurable cognitive gains into changes in the learning process and practice
• Consider organisational issues in the adoption of mobile learning practice and its integration with existing practices and understand how this integration affects attributes of in/formality
• Span the lifecycle of the mobile learning innovation that is evaluated, from conception to full deployment and beyond.
The evaluation framework of Myartspace consisted of three levels:
1. Micro level: the micro level examines the individual activities of the technology users and assesses usability and utility. In the case of Myartspace the activities included collecting objects through exhibit codes, making notes, contacting people who have collected a particular item, recording audio, and taking pictures.
2. Meso level: the meso level examines the learning experience as a whole, to identify learning breakthroughs and breakdowns; it also examines how well the learning experience integrates with other related learning experiences. In the case of Myartspace, evaluation at this level involved exploring whether there was a successful connection between learning in the museum and in the classroom as well as identifying critical incidents that reveal new patterns and forms of learning or where learning activity is impeded.
3. Macro level: the macro level examines the longer term impact of the new technology on established educational and learning practice. For Myartspace this related to the organisation of school museum visits. The evaluation at this level looked, for example, at the appropriation of the new technology by teachers, the emergence of new museum practices in supporting school visits, and how they related to the original project visions.
For Myartspace, the development comprised four phases:
(1) Requirements analysis, to establish the requirements for the socio-technical system (people and their interactions with technology) and specify how it would work, through consultation with the different stakeholder groups;
(2) Design of the user experience and interface;
(3) Implementation of the service; and
(4) Deployment of the service.
To establish the value of the service at the three levels, the evaluation framework explores the gap between expectations and reality and also unforeseen processes and outcomes.
This happened in three stages:
1. Stage 1: explores what is supposed to happen at a level. User expectations at each level can be captured through interviews with users (e.g. teachers, students, museum staff) and by analysing user documentation, training sessions and materials.
2. Stage 2: observes what actually happened at a level. The user experience is documented to establish the reality of technology use for the different users.
3. Stage 3: examines the gaps between user expectations and reality through a combination of reflective interviews with users and critical analysis of the findings from stages 1 and 2 by the evaluators.
References that I may want to read further in the future:
Kjeldskov, J. & J. Stage (2004). "New Techniques for Usability Evaluation of Mobile Systems." International Journal of Human-Computer Studies 60: 599-620.
Meek, J. (2006). Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology. Unpublished PhD thesis, School of Electronic, Electrical and Computer Engineering, The University of Birmingham.
Vavoula, G., J. Meek, M. Sharples, P. Lonsdale & P. Rudman (2006a). A lifecycle approach to evaluating MyArtSpace. 4th International Workshop of Wireless, Mobile and Ubiquitous Technologies in Education (WMUTE 2006 ), Athens, Greece, IEEE Computer Society.
So now...I have finished reading on "Usability" from articles in UsabilityMLearnELearn_20090821 folder.
Giasemi N. Vavoula. University of Leicester, 105 Princess Road East, Leicester, LE1 7LG, UK. gv18@leicester.ac.uk Tel: +44 (0) 116 2523966
Mike Sharples. LSRI, University of Nottingham, Exchange Building, Jubilee Campus, Wollaton Road, Nottingham, NG8 1BB, UK. mike.sharples@nottingham.ac.uk Tel: +44 (0) 115 9513716
Published in Proceedings of Mlearn2008.
ABSTRACT
We propose six challenges in evaluating mobile learning: capturing and analysing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing in/formality. A three-level framework for evaluating mobile learning is presented, comprising a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organisational contexts. The paper concludes with a discussion of how the framework meets the evaluation challenges and with suggestions for further extensions.
We now appreciate mobile learning not just as learning that is facilitated by mobile technology, but also as the processes of coming to know through conversations and explorations across multiple contexts amongst people and personal interactive technologies (Sharples et al. 2007b). Such evolving conceptions introduce numerous challenges to all aspects of mobile learning research, including evaluation. As the field matures, our frameworks and tools need to respond to these challenges.
CHALLENGE 1: CAPTURING LEARNING CONTEXT AND LEARNING ACROSS CONTEXTS
A major task for educational evaluation is to capture and analyse learning in context.
To appreciate the challenge, let us compare mobile learning contexts with traditional classroom contexts from the researcher’s perspective.
In order to establish and document the learning context, the researcher needs to know:
the location of learning and the layout of the space (where); the social setting (who, with whom, from whom); the learning objectives and outcomes (why and what); the learning method(s) and activities (how); and the learning tools (how).
CHALLENGE 2: HAS ANYONE LEARNED ANYTHING?
A second challenge that faces mobile learning evaluation is the assessment of learning processes and outcomes.
In traditional learning settings such as the classroom there are well-established and accepted methods for the assessment of learning activities, such as essay writing, multiple choice tests, open-book exams, and unseen examinations.
Distinctions have been made between formative assessment (aiming to provide students with feedback regarding their progress) and summative assessment (aiming to judge and sum up the students’ achievements) (Scriven 1967)...
CHALLENGE 3: AN ETHICAL QUESTION
Mobile learning can involve the use of mobile technology, which may also be personal technology. Tapping into a person’s mobile phone to find out how they have been using it to learn might mean invading that person’s privacy.
Although research ethics frameworks are implemented by most research institutions and organisations, mobile learning research raises profound ethical issues.
CHALLENGE 4: LET’S NOT FORGET THE TECHNOLOGY
Evaluations of mobile learning often reference inherent limitations of mobile devices, such as their small screens, short battery lives, intermittent connectivity, and associated human factors, all of which affect their usability. As the focus of research shifts from the mobility of the technology to the mobility of the learner, additional issues arise as learners move across multiple devices, possibly over short time periods in multiple locations. Assessing the usability of the mobile technology and the effectiveness of its integration with the mobile learning practice remains a high priority for evaluation.
CHALLENGE 5: SEEING THE BIGGER PICTURE
Oliver and Harvey (2002) suggest four different kinds of impact of educational technologies in Higher Education (HE): impact on students’ learning, impact on individual academics’ practice, impact on institution, and national impact.
In the same context of HE Price and Oliver (2007) identify three types of impact studies: anticipatory, ongoing and achieved.
Anticipatory studies relate to pre-intervention intentions, opinions and attitudes; ongoing studies focus on analysing processes of integration; and achieved studies are summative studies of technology no longer ‘novel’.
Riley (2007) extends this impact framework by distinguishing between minor modifications and culturally significant changes in practice, and suggesting that different kinds of change will emerge over different timescales.
Although not exclusively linked with HE contexts, mobile learning evaluation has similar questions to answer regarding impact.
CHALLENGE 6: FORMAL OR INFORMAL?
Mobile learning is often defined in terms of the technology that mediates the learning experience: if the technology is mobile, so is the learning. Mobility, however, is not an exclusive property of the technology, it also resides in the lifestyle of the learner, who in the course of everyday life moves from one context to another, switching locations, social groups, technologies and topics; and learning often takes place inconspicuously or is crammed in the short gaps between these transitions. Although this view of learning is inclusive of formal education contexts, it is particularly pertinent to everyday, informal learning.
Nevertheless, characterising a learning experience as formal or informal can be complicated. For example, is the learning of pupils visiting a museum (largely considered an informal learning setting) with their school (an irrefutably formal learning setting) a case of formal or informal learning?
REQUIREMENTS FOR MOBILE LEARNING EVALUATION
The challenges discussed in the previous sections indicate that mobile learning evaluation needs to:
• Capture and analyse learning in context with consideration of learner privacy
• Assess the usability of the technology and how it affects the learning experience
• Look beyond measurable cognitive gains into changes in the learning process and practice
• Consider organisational issues in the adoption of mobile learning practice and its integration with existing practices and understand how this integration affects attributes of in/formality
• Span the lifecycle of the mobile learning innovation that is evaluated, from conception to full deployment and beyond.
The evaluation framework of Myartspace consisted of three levels:
1. Micro level: the micro level examines the individual activities of the technology users and assesses usability and utility. In the case of Myartspace the activities included collecting objects through exhibit codes, making notes, contacting people who have collected a particular item, recording audio, and taking pictures.
2. Meso level: the meso level examines the learning experience as a whole, to identify learning breakthroughs and breakdowns; it also examines how well the learning experience integrates with other related learning experiences. In the case of Myartspace, evaluation at this level involved exploring whether there was a successful connection between learning in the museum and in the classroom as well as identifying critical incidents that reveal new patterns and forms of learning or where learning activity is impeded.
3. Macro level: the macro level examines the longer term impact of the new technology on established educational and learning practice. For Myartspace this related to the organisation of school museum visits. The evaluation at this level looked, for example, at the appropriation of the new technology by teachers, the emergence of new museum practices in supporting school visits, and how they related to the original project visions.
For Myartspace, the development comprised four phases:
(1) Requirements analysis, to establish the requirements for the socio-technical system (people and their interactions with technology) and specify how it would work, through consultation with the different stakeholder groups;
(2) Design of the user experience and interface;
(3) Implementation of the service; and
(4) Deployment of the service.
To establish the value of the service at the three levels, the evaluation framework explores the gap between expectations and reality and also unforeseen processes and outcomes.
This happened in three stages:
1. Stage 1: explores what is supposed to happen at a level. User expectations at each level can be captured through interviews with users (e.g. teachers, students, museum staff) and by analysing user documentation, training sessions and materials.
2. Stage 2: observes what actually happened at a level. The user experience is documented to establish the reality of technology use for the different users.
3. Stage 3: examines the gaps between user expectations and reality through a combination of reflective interviews with users and critical analysis of the findings from stages 1 and 2 by the evaluators.
References that I may want to read further in the future:
Kjeldskov, J. & J. Stage (2004). "New Techniques for Usability Evaluation of Mobile Systems." International Journal of Human-Computer Studies 60: 599-620.
Meek, J. (2006). Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology. Unpublished PhD thesis, School of Electronic, Electrical and Computer Engineering, The University of Birmingham.
Vavoula, G., J. Meek, M. Sharples, P. Lonsdale & P. Rudman (2006a). A lifecycle approach to evaluating MyArtSpace. 4th International Workshop of Wireless, Mobile and Ubiquitous Technologies in Education (WMUTE 2006 ), Athens, Greece, IEEE Computer Society.
So now...I have finished reading on "Usability" from articles in UsabilityMLearnELearn_20090821 folder.
Labels:
Kjeldskov,
Lonsdale,
Meek,
mobile learning,
Rudman,
Sharples,
Stage,
usability evaluation method,
Vavoula
Wednesday, August 26, 2009
Aug 26 - Vainio & Ahonen, A critical approach to an adaptive user interface design
A critical approach to an adaptive user interface design.
Teija Vainio. Hypermedia Laboratory, University of Tampere, 33014 University of Tampere, Finland. E-mail: teija.vainio@uta.fi
Mikko Ahonen. Hypermedia Laboratory, University of Tampere, 33014 University of Tampere, Finland. E-mail: mikko.ahonen@uta.fi
Learning With Mobile Devices: Reserch and Development. a book of papers, edited by Jill Attewell and Carol Savill-Smith. Learning and Skills Development Agency, Regent Arcade House19–25 Argyll Street, London W1F 7LS. pg 189-192
Abstract:
In our study we present a critical approach to usability issues and usability evaluation particularly involving adaptive user interface design for mobile learning environments. We describe some design challenges for adaptive user interfaces and key usability issues in the MOBIlearn project. In addition, we compare these challenges in three different learning contexts.
Our hypothesis is that the concept of ‘learning’ is not clearly defined in most usability studies. Additionally, the influences of different learning contexts are not considered enough in usability evaluation. Our aim is to develop mobile systems that are adaptable to more than one learning context.
This paper introduces one approach to a user interface design in the MOBIlearn project, eg adaptive human interface design. Our work aims to design user interfaces that take account
of different kind of users, contexts, content and devices.
..the learning itself is different in each of these contexts: the first is focused on blended, formal learning, the second more on informal learning and the last on lifelong learning.
Studies of web-based educational systems and mobile learning environments have recently started to focus on systems described as adaptive. Adaptive systems can be seen as an alternative to the ‘one-size-fits-all’ approach (Brusilowsky et al. 2000).
In many cases adaptation is seen as a part of contextawareness in mobile applications and systems.
Adaptivity refers to a system that adapts itself according to the user. Furthermore, adaptability refers to a system where the users have to change the system behaviour.
Adaptive user interfaces can be focused on, for example, information or content-based filtering,
recommendation, social or collaborative filtering, or optimising (Langley 1999).
Since the learning systems with mobile devices are expected to be used for the short-term, we argued that it might be better let the system adapt itself (adaptivity) to the user rather than forcing the users to change the system behaviour (adaptability).
We see adaptive user interface design as a vital part of the usability design of an application or a service.
Usefulness is defined as the ability to achieve a goal by using the system. Usefulness can be divided into utility and usability. Utility refers to the functionality of a system, and usability to the ability to make use of that functionality (Nielsen 1995).
The usability of a system is determined by the usability of both the underlying system engine and the contents and structure of the information base. Usability has been associated with five
attributes: ease of learning, efficiency, easy to remember, consistency (few errors) and pleasantness of use (Nielsen 1995).
Usability focuses on making applications easy for people to use. Accessibility on the other hand focuses on making applications equally easy for everyone to use, including people with a disability. ... In the context of mobile collaboration, access and accessibility are often more essential issues than ease of use (Ahonen 2003).
Vavoula and Sharples (2002) have inspected learning episodes and personal learning projects, and have developed the following criteria for lifelong learning organisers (LLOs): LLOs should be available and functional any time, during any day of the week.
My Comments: My PhD research may also go on "adaptive" concept; generalisation; one size fits all; one set of usability criteria fits all MLearn applications.
References to be considered for future reading:
MOBIlearn (2003). MOBIlearn EU IST project overview. At http://www.mobilearn.org/, accessed January 2004.
Nielsen J (1995). Multimedia and hypertext. the internet and beyond. Boston: AP Professional, 280–281.
Vavoula G, Sharples M (2002). Requirements for the design of lifelong learning organisers. S Anastopoulou, M Sharples, Vavoula (eds) MLearn 2002: Proceedings of the European Workshop on Mobile and Contextual Learning. Birmingham: 23–26.
Teija Vainio. Hypermedia Laboratory, University of Tampere, 33014 University of Tampere, Finland. E-mail: teija.vainio@uta.fi
Mikko Ahonen. Hypermedia Laboratory, University of Tampere, 33014 University of Tampere, Finland. E-mail: mikko.ahonen@uta.fi
Learning With Mobile Devices: Reserch and Development. a book of papers, edited by Jill Attewell and Carol Savill-Smith. Learning and Skills Development Agency, Regent Arcade House19–25 Argyll Street, London W1F 7LS. pg 189-192
Abstract:
In our study we present a critical approach to usability issues and usability evaluation particularly involving adaptive user interface design for mobile learning environments. We describe some design challenges for adaptive user interfaces and key usability issues in the MOBIlearn project. In addition, we compare these challenges in three different learning contexts.
Our hypothesis is that the concept of ‘learning’ is not clearly defined in most usability studies. Additionally, the influences of different learning contexts are not considered enough in usability evaluation. Our aim is to develop mobile systems that are adaptable to more than one learning context.
This paper introduces one approach to a user interface design in the MOBIlearn project, eg adaptive human interface design. Our work aims to design user interfaces that take account
of different kind of users, contexts, content and devices.
..the learning itself is different in each of these contexts: the first is focused on blended, formal learning, the second more on informal learning and the last on lifelong learning.
Studies of web-based educational systems and mobile learning environments have recently started to focus on systems described as adaptive. Adaptive systems can be seen as an alternative to the ‘one-size-fits-all’ approach (Brusilowsky et al. 2000).
In many cases adaptation is seen as a part of contextawareness in mobile applications and systems.
Adaptivity refers to a system that adapts itself according to the user. Furthermore, adaptability refers to a system where the users have to change the system behaviour.
Adaptive user interfaces can be focused on, for example, information or content-based filtering,
recommendation, social or collaborative filtering, or optimising (Langley 1999).
Since the learning systems with mobile devices are expected to be used for the short-term, we argued that it might be better let the system adapt itself (adaptivity) to the user rather than forcing the users to change the system behaviour (adaptability).
We see adaptive user interface design as a vital part of the usability design of an application or a service.
Usefulness is defined as the ability to achieve a goal by using the system. Usefulness can be divided into utility and usability. Utility refers to the functionality of a system, and usability to the ability to make use of that functionality (Nielsen 1995).
The usability of a system is determined by the usability of both the underlying system engine and the contents and structure of the information base. Usability has been associated with five
attributes: ease of learning, efficiency, easy to remember, consistency (few errors) and pleasantness of use (Nielsen 1995).
Usability focuses on making applications easy for people to use. Accessibility on the other hand focuses on making applications equally easy for everyone to use, including people with a disability. ... In the context of mobile collaboration, access and accessibility are often more essential issues than ease of use (Ahonen 2003).
Vavoula and Sharples (2002) have inspected learning episodes and personal learning projects, and have developed the following criteria for lifelong learning organisers (LLOs): LLOs should be available and functional any time, during any day of the week.
My Comments: My PhD research may also go on "adaptive" concept; generalisation; one size fits all; one set of usability criteria fits all MLearn applications.
References to be considered for future reading:
MOBIlearn (2003). MOBIlearn EU IST project overview. At http://www.mobilearn.org/, accessed January 2004.
Nielsen J (1995). Multimedia and hypertext. the internet and beyond. Boston: AP Professional, 280–281.
Vavoula G, Sharples M (2002). Requirements for the design of lifelong learning organisers. S Anastopoulou, M Sharples, Vavoula (eds) MLearn 2002: Proceedings of the European Workshop on Mobile and Contextual Learning. Birmingham: 23–26.
Tuesday, August 25, 2009
Aug 25 - Vavoula et al, A Lifecycle approach to evaluating MyArtSpace
Giasemi Vavoula, Julia Meek, Mike Sharples, Peter Lonsdale, Paul Rudman. A Lifecycle approach to evaluating MyArtSpace.
Proceedings: Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education. WMUTE 2006
The Lifecycle Evaluation Approach [2][3] places evaluation at the centre of the development process, from the very early stages of design to the final assessment of the technology. The approach draws on evaluation methods and ideas from software engineering, educational evaluation, and models for evaluating learning technology. Evaluation activities are undertaken at key points in the lifecycle of the system design process.
The overall goal of the evaluationwas agreed by the key stakeholders to be the assessment of the service’s usability, accessibility, effectiveness, and capability to engage the user.
The requirements (112 in total) that emerged from the user consultation workshop were divided into six main categories:
1. Teacher requirements: requirements for designing the teacher experience
2. Student requirements: requirements for designing the student experience
3. Museum requirements: requirements for designing the museum staff experience and
also institutional requirements
4. Culture Online requirements: requirements relevant to the funding body, pertaining to
overall project objectives, qualitative and quantitative measurement criteria, etc.
5. General system requirements: all other requirements that did not directly concern a
specific group
6. Usability requirements: requirements for designing an easy to use, easy to learn and enjoyable service/system for all user groups.
Due to the number of requirements that had been generated, the representatives were divided into groups and each group was given a subset of requirements to review and was asked to verify that each requirement was unambiguous and testable.
Subsequently, the stakeholders were asked to use the MoSCoW [4] technique from Dynamic Systems Development Method to indicate that a requirement is a:
MUST: must have this
SHOULD: should have this if at all possible
COULD: could have this if it does not affect anything else
WOULD: won’t have this time but would like to have in the future
The outcomes of the discussion in the subgroups were then presented to the larger group, who finalised the categorisation and prioritisation.
The workshop was successful in enabling the requirements to be discussed, clarified and prioritised. The outcome was an agreed set of user requirements and benchmarks for both the Web portal of MyArtSpace and the Mobile phone service.
MyArtSpace is a service that makes use of different technologies (mobile phones and the Web) to deliver a learning experience that traverses the museum and the
classroom. It was therefore essential that the evaluation of the service did not stop at the evaluations of its individual components; rather, it had to go beyond the technology to look at the learning experience as a whole. The Lifecycle Evaluation Approach has made this possible through its focus on both the usability and the pedagogy of MyArtSpace, which enabled a full picture of the use of a new educational technology to be understood, assessed and improved.
References that I would like to read further in the future:
[2] Meek, J. (2006) Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology. Unpublished PhD Thesis, The University of Birmingham, UK.
[3] Meek, J. and Sharples, M. (2001) A lifecycle approach to the evaluation of learning technology. Proceedings of CAL2001 Conference, Warwick, 2-1 April, pp. 195-196.
[4] http://www.dsdm.org/
Proceedings: Fourth IEEE International Workshop on Wireless, Mobile and Ubiquitous Technology in Education. WMUTE 2006
The Lifecycle Evaluation Approach [2][3] places evaluation at the centre of the development process, from the very early stages of design to the final assessment of the technology. The approach draws on evaluation methods and ideas from software engineering, educational evaluation, and models for evaluating learning technology. Evaluation activities are undertaken at key points in the lifecycle of the system design process.
The overall goal of the evaluationwas agreed by the key stakeholders to be the assessment of the service’s usability, accessibility, effectiveness, and capability to engage the user.
The requirements (112 in total) that emerged from the user consultation workshop were divided into six main categories:
1. Teacher requirements: requirements for designing the teacher experience
2. Student requirements: requirements for designing the student experience
3. Museum requirements: requirements for designing the museum staff experience and
also institutional requirements
4. Culture Online requirements: requirements relevant to the funding body, pertaining to
overall project objectives, qualitative and quantitative measurement criteria, etc.
5. General system requirements: all other requirements that did not directly concern a
specific group
6. Usability requirements: requirements for designing an easy to use, easy to learn and enjoyable service/system for all user groups.
Due to the number of requirements that had been generated, the representatives were divided into groups and each group was given a subset of requirements to review and was asked to verify that each requirement was unambiguous and testable.
Subsequently, the stakeholders were asked to use the MoSCoW [4] technique from Dynamic Systems Development Method to indicate that a requirement is a:
MUST: must have this
SHOULD: should have this if at all possible
COULD: could have this if it does not affect anything else
WOULD: won’t have this time but would like to have in the future
The outcomes of the discussion in the subgroups were then presented to the larger group, who finalised the categorisation and prioritisation.
The workshop was successful in enabling the requirements to be discussed, clarified and prioritised. The outcome was an agreed set of user requirements and benchmarks for both the Web portal of MyArtSpace and the Mobile phone service.
MyArtSpace is a service that makes use of different technologies (mobile phones and the Web) to deliver a learning experience that traverses the museum and the
classroom. It was therefore essential that the evaluation of the service did not stop at the evaluations of its individual components; rather, it had to go beyond the technology to look at the learning experience as a whole. The Lifecycle Evaluation Approach has made this possible through its focus on both the usability and the pedagogy of MyArtSpace, which enabled a full picture of the use of a new educational technology to be understood, assessed and improved.
References that I would like to read further in the future:
[2] Meek, J. (2006) Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology. Unpublished PhD Thesis, The University of Birmingham, UK.
[3] Meek, J. and Sharples, M. (2001) A lifecycle approach to the evaluation of learning technology. Proceedings of CAL2001 Conference, Warwick, 2-1 April, pp. 195-196.
[4] http://www.dsdm.org/
Aug 25 - Ahonen, Accessibility Challenges with Mobile Lifelong Learning Tools...
Mikko AHONEN, Accessibility challenges with mobile lifelong learning tools and related collaboration. University of Tampere, Hypermedia Laboratory. mikko.ahonen@uta.fi
Usability focuses on making applications and websites easy for people to use.
Accessibility focuses on making them equally easy for everyone to use,
including people with a disability. (WAI, 2003)
When mobile tools exchanges data with other devices via a potentially unreliable or narrow communication line, as in wireless communications, user interfaces for management of database communications of mobile tools are required to meet user's needs such as fast response, high usability, reliability and easy-to-use features. Standardizing these new user-interfaces will be very beneficial and convenient for mobile users.
When Blanchard (2001) took a look at usability standards like ISO 13407
HUMAN CENTRED DESIGN PROCESSES FOR INTERACTIVE SYSTEMS, he noticed that
focus is on usability and not accessibility per se and there were no statements
requiring systems to be accessible.
Ahonen's original interest on accessibility issues has come up in the development of
evaluation criteria for mobile learning solutions. Action: Read about more of Ahonen's works.
Churchill and Wakeford (2002, 173) have foreseen this challenge area:
“Experience of mobility is embedded in an experience of temporality which includes mutually negotiated rhythms of contact, availability and accessibility.”
They have suggested a following design framework for collaboration on the move with two dimensions:
1. Tight versus Loose Mobility
2. Close versus Distant Information
References that I may want to read further in the future:
Ahonen, M. (2003) Mobility, Accessibility and Learning. Keynote Speech in Mlearn 2003 Conference. London.
Blanchard, H. E. (2001). International standards: Their scope, quality, and impact. In M. J. Smith & G. Salvendy (Eds.) Systems, social and internationalization design aspects of human-computer interaction. pp. 537-540. Erlbaum. Mahwah, NJ.
Churchill, E. & Wakeford, N. (2002) Framing Mobile Collaborations and Mobile Technologies. In Brown, B., Green, N. & Harper, R. (Eds.) Wireless World. Social and Interactional Aspects of the Mobile Age. Springer. London.
ISO (2003) ISO/IEC 18021, Information technology - User interfaces for Mobile tools for management of database communications in a client-server model. First Edition. International standard specification. Available online.
Vavoula, G. & Sharples, M. (2002) Requirements for the Design of Lifelong Learning Organisers. Mlearn 2002 Conference Proceedings. London.
WAI (2003) Web Accessibility Initiative (WAI). Guidelines working document. World Wide Web Consortium. Online: http://www.w3.org/WAI/
Usability focuses on making applications and websites easy for people to use.
Accessibility focuses on making them equally easy for everyone to use,
including people with a disability. (WAI, 2003)
When mobile tools exchanges data with other devices via a potentially unreliable or narrow communication line, as in wireless communications, user interfaces for management of database communications of mobile tools are required to meet user's needs such as fast response, high usability, reliability and easy-to-use features. Standardizing these new user-interfaces will be very beneficial and convenient for mobile users.
When Blanchard (2001) took a look at usability standards like ISO 13407
HUMAN CENTRED DESIGN PROCESSES FOR INTERACTIVE SYSTEMS, he noticed that
focus is on usability and not accessibility per se and there were no statements
requiring systems to be accessible.
Ahonen's original interest on accessibility issues has come up in the development of
evaluation criteria for mobile learning solutions. Action: Read about more of Ahonen's works.
Churchill and Wakeford (2002, 173) have foreseen this challenge area:
“Experience of mobility is embedded in an experience of temporality which includes mutually negotiated rhythms of contact, availability and accessibility.”
They have suggested a following design framework for collaboration on the move with two dimensions:
1. Tight versus Loose Mobility
2. Close versus Distant Information
References that I may want to read further in the future:
Ahonen, M. (2003) Mobility, Accessibility and Learning. Keynote Speech in Mlearn 2003 Conference. London.
Blanchard, H. E. (2001). International standards: Their scope, quality, and impact. In M. J. Smith & G. Salvendy (Eds.) Systems, social and internationalization design aspects of human-computer interaction. pp. 537-540. Erlbaum. Mahwah, NJ.
Churchill, E. & Wakeford, N. (2002) Framing Mobile Collaborations and Mobile Technologies. In Brown, B., Green, N. & Harper, R. (Eds.) Wireless World. Social and Interactional Aspects of the Mobile Age. Springer. London.
ISO (2003) ISO/IEC 18021, Information technology - User interfaces for Mobile tools for management of database communications in a client-server model. First Edition. International standard specification. Available online.
Vavoula, G. & Sharples, M. (2002) Requirements for the Design of Lifelong Learning Organisers. Mlearn 2002 Conference Proceedings. London.
WAI (2003) Web Accessibility Initiative (WAI). Guidelines working document. World Wide Web Consortium. Online: http://www.w3.org/WAI/
Labels:
Ahonen,
Blanchard,
Churchill,
ISO/IEC18021,
mobile learning,
mobility,
Sharples,
usability,
Vavoula,
WAI
Subscribe to:
Posts (Atom)