Sep 4 - Costa et al, Evaluating Web Usability Using Small Display Devices

Evaluating Web Usability Using Small Display Devices.
Carlos J. Costa. DCTI – ISCTE, Adetti/ISCTE, Lisboa, Portugal. carlos.costa@iscte.pt
José P. Novais Silva. Portugal Telecom, Lisboa, Portugal. jose.p.silva@telecom.pt
Manuela Aparício. ITML, Lisboa, Portugal. manuela@design.itml.org

SIGDOC’07, October 22–24, 2007, El Paso, Texas, USA.

ABSTRACT
Usability is an important issue studied by many researchers and practitioners. Specifically, we are interested in evaluating web usability using small display devices. In this paper we evaluate different type of navigation using small display devices: links, folio and search. In order to evaluate it, we propose a laboratory prototype.


Mobile devices enable users to access information and web-based services from any location either by PC or by small display devices. But, those mobile devices are restricted by small screen size, which limit the amount of information that can be displayed at same time. Therefore, it has become increasingly important to learn how to evaluate its use and how to design mobile devices functionalities. It is already proven that an attractive interface is not necessarily good and usable; therefore “a good graphic design and attractive displays can increase users´ satisfaction and thus improve productivity” [10].

Starting in the mid of 80s and gaining strength in the 90s, the interface development community employed usability engineering methods to design and test software systems for ease of use, ease of learning, memorability, lack of errors, and satisfaction [17].
Usability practitioners of the 1990s considered two factors as measures of usability [3]: the ease of learning and the ease of use.
Learnability, flexibility and robustness are pointed out as three principles that support usability [10].
In this study there were taken into account learnability and flexibility. As learnability [10] is intended to be “the ease with which new users can begin effective interaction and achieve maximal performance. Flexibility is the multiplicity of ways in which the user and system exchange information.”

Usability testing is an usual tool used to evaluate the usability of a mobile application in a development process. In fact, mobile and handheld usability testing could be even more important than computer-based usability testing.
Webcredible [23] explain the main reasons for this:
• Increase of small display devices sales.
• Less experience of using small display devices to navigate on the web than using desktop computers.

USABILITY TESTING IN SMALL DISPLAY DEVICES
What is the criterion that is to be taken into account when analysing usability? The answer to the question is mentioned by various authors [2][12][17][20].
Focusing on the user, traditional usability can be characterized by the following, [17]: Learnability, Memorability, Efficiency, Errors and Satisfaction.
However, Seffah and colleagues [20] presents another concept for usability. Combining the various standards and models (like Nielson usability characteristics) they unified then into a single model of usability measurement. This model called Quality in Use Integrated Measurement (QUIM), include 10 usability factors: 1-Efficiency; 2-Effectiveness; 3- Productivity; 4-Satisfaction; 5-Learnability; 6-Safety; 7- Trustfulness; 8-Accessibility; 9-Universality; 10-Usefulness.

How usability can be tested? ...
Lee and Grice [15] propose a usability testing that combines heuristics, questionnaires, and scenarios for developing mobile applications.
According to Andrews [1] there are three usability inspection methods: Heuristic Evaluation, Cognitive Walkthrough and Action Analysis. In Heuristic Evaluation a small set of evaluators examines interface and judges its compliance with recognized usability principles, while in Cognitive Walkthrough, exist a taskoriented walkthrough based on formal cognitive model of user behaviour (analyses learnability).
On the other and, in Action Analysis is made an quantitative analysis of actions to predict time required for tasks, based on time estimates for typical interface actions (analyses efficiency).

WEB NAVIGATION
The researcher collects user's actions, for future use. Watters and Mackay [22] identified different feature items for each kind of navigation: scrolling, menu/options and search features.
Scrolling has the following characteristics [21][23]:
• Scrolling slows time to complete tasks;
• Scrolling does not necessarily increases error rate;
• Scrolling requires use of stylus on PDA;
• Horizontal scrolling is hard for reading and comparison tasks;
• Vertical scrolling is adequate for skimming and scanning tasks;
• Paging gives better performance for users than scrolling;
Menus/Options (links and folders) have the following characteristics [21][23]:
• Feedback decreases error rate in menu selection
• Performance is better on shallow hierarchies
Search Features have the following characteristics[21][23]:
• Search used for text and lists
• There is an improvement over past search
• Breaking lists does not affect search effectiveness
• Small screen devices users’ tend to use search often
• Search on small screen devices in tables may deteriorate performance.

4 Navigation Types
Figure 2. “Link” navigation with the two zones;
Figure 3. “Scroll” navigation;
Figure 4. “Folio” navigation;
Figure 5. “Search” navigation.

CONCLUSIONS
In this paper we presented two main contributions from a research.
The first one is a tool, employed to help users evaluation navigation type implemented in sites that are accessed by small display devices.
The second contribution results from the use of this tool.
In fact, we evaluate different type of navigation using small display devices. Those types of navigation are links, folio and search.
We concluded that each one Links is the more efficient and also the more effective type of navigation. But, results are not conclusive in what concerns effectiveness. On the other hand,
preferences do not mach with effectiveness and efficiency.
Results presented here are legitimate for the specific situation presented here. For example, if the number of sites is enlarged for thousands, scroll may not be the more adequate type of navigation. On the other hand, the font, the character size and the number of words shown in each line or in each screen may also influence results.

References that I may want to read further in future:
[12] Kaikkonen, A., Kallio, T., Kekalainen, A., Kankainen, A. and Cankar, M. “Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing”, Journal of Usability Studies, Issue 1, Vol. 1, November 2005, pp. 4-16
[15] Lee, K., Grice, R. A. “Developing a New Usability Testing Method for Mobile Devices”, Proceedings International Professional Communication Conference, 2004. IPCC, 2004.
[19] Qiu, M, Zhang, K and Huang, M. Usability in mobile interface browsing, Web Intelligence and Agent Systems: An International Journal, 3, IOS Press 2005, pp. 1-18..
[20] Seffah, A., Donyaee, M., Kline, R. B., Padda, H. K. “Usability measurement and metrics: A consolidated model”, Software Quality Control Archive Volume 14 , Issue 2 (June 2006), Kluwer Academic Publishers, 2006
[21] Silva, J. P.”The Web as usability testing method for small display devices” in C. Costa (ed.) Information Technology, Organizations and Teams; press.itml.org, 2007, pp. 131-142.
[23] Webcredible , The usability and accessibility specialists “Mobile & handheld usability testing - why it matters”, June 2006. 268