Aug 29 - Jokela et al, ..Standard Definition of Usability: Analyzing ISO 13407 against ISO

The Standard of User-Centered Design and the Standard Definition of Usability: Analyzing ISO13407 against ISO9241-11.
Timo Jokela, Netta Iivari. Oulu University, P.O. Box 3000, 90014 Oulu, Finland. +358 8 5531011 {timo.jokela, mailto:netta.iivari%7D@oulu.fi
Juha Matero, Minna Karukka. Nokia, P.O. Box 50, 90571 Oulu, Finland. {juha.p.matero, mailto:minna.karukka%7D@nokia.com

ABSTRACT
ISO 9241-11 and ISO 13407 are two important standards related to usability: the former one provides the definition of usability and the latter one guidance for designing usability. We carried out an interpretative analysis of ISO 13407 from the viewpoint of the standard definition of
usability from ISO 9241-11. The results show that ISO 13407 provides only partly guidance for designing usability as presumed by the definition. Guidance for describing users and environments are provided but very limited guidance is provided for the descriptions of user goals and usability measures, and generally for the process of producing the various outcomes.

Probably the best known definition of usability is by Nielsen: usability is about learnability, efficiency, memorability, errors, and satisfaction [16].

However, the definition of usability from ISO 9241-11 (Guidance on usability) [11] – “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” - is becoming the main reference of usability.

In addition that it is largely recognized in literature, this ‘standard’ definition of usability is used in the recent Common Industry Format, CIF, for usability testing [1].

To improve the usability of software and information systems, the paradigm of user-centered design1, UCD, has been proposed by a number of method and methodology books, starting from Nielsen [16] to ones published in late 90’s, [8], [4], [5], [15] and ending up with a set of very recent ones, [17] and [18].

1 Called ‘human-centered design’ in ISO 13407. Also called ‘usability engineering’.

My Comments: I think this article is a good resource for "Definition of Usability."

ISO 13407 [9], Human-centred design processes for interactive systems, is a standard that provides guidance for user-centered design. ...it describes usability at a level of principles, planning and activities. A third important aspect is that ISO 13407 explicitly uses the standard definition of usability from ISO 9241-11 as a reference for usability.

Usability is defined in ISO 9241-11 [11] as follows:
Usability: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.
The terms are further defined as follows:
Effectiveness: the accuracy and completeness with which users achieve specified goals
Efficiency: the resources expended in relation to the accuracy and completeness with which users achieve goals
Satisfaction: freedom from discomfort, and positive attitude to the use of the product
Context of use: characteristics of the users, tasks and the organizational and physical environments
Goal: intended outcome
Task: activities required to achieve a goal

Generally, this definition of usability is a ‘broad’ approach to usability [2]: usability is about supporting users in achieving their goals in their work, it is not only a characteristic of a user interface.

..usability is a function of users of a product or a system (specified users). Further, for each user, usability is a function of achieving goals in terms of a set of attributes (i.e. effectiveness, efficiency and satisfaction) and environment of use.

As an example, one usability measure of a bank machine
could be: • 90 % users achieve the goal (Es) in less than 1 minute (Ey) with an average satisfaction rating ‘6’ (S) when users are novice ones (U), and they want to have a desired sum of cash withdrawn (G) with any bank machine (Et).

The analysis of the definition of usability shows that one needs to determine the following outocomes when the definition is used in a development project:
(1) The users of the system,
(2) Goals of users,
(3) Environments of use
(4) Measures of effectiveness, efficiency and satisfaction.

ISO 13407 is aimed to provide ‘overview’ guidance for the planning and management of user-centered design, not to provide detailed coverage of the methods and technique.

ISO 13407 is an international standard established in 1999. The standard “provides guidance on humancentred design activities throughout the life cycle of computer-based interactive systems”. The standard aims at “those managing design processes” and does not provide detailed coverage of methods and techniques.

ISO 13407 describes user-centered design from four
different aspects:
• Rationale for UCD
• Planning UCD
• Principles of UCD
• Activities of UCD.

Rationale. The rationale part briefly describes the benefits that usable systems provide, such as reduction of training and support costs, improved user satisfaction and productivity of users.

Principles. The standard identifies four general principles that characterize user-centered design, and that are not bound to any specific phase of development cycle:
• The active involvement of users and a clear understanding of user and task requirements
• An appropriate allocation of functions between users and technology
• Iteration of design solutions
• Multi-disciplinary design.

Planning. The planning part provides guidance in fitting user-centered design activities into the overall system development process. Among other things, the standard emphasizes that project plans should reserve time and resources for iteration and user feedback. The importance of teamwork and communication is also mentioned.

Activities. The core of the standard – stated explicitly– is the description of user-centered design activities. The standard identifies four main activities of UCD, illustrated
in Figure 2:

Understand and Specify Context of Use. Know the user, the environment of use, and the tasks that he or she uses the product for.
Specify the User and Organizational Requirements. Determine the success criteria of usability for the product in terms of user tasks, e.g. how quickly a typical user should be able to complete a task with the product.
Determine the design guidelines and constraints. Produce Design Solutions. Incorporate HCI knowledge (of visual design, interaction design, usability) into design solutions.
Evaluate Designs against Requirements. The usability of designs is evaluated against user tasks.

Usability is one type of a quality characteristic in a product [10] among others, such as functionality, efficiency, reliability, maintainability and portability. In the requirement phase, when the quality requirements for a product are determined, also the usability requirements should be determined.
While all activities of life-cycle are relevant in the design of usability, the definition of usability has a critical impact especially in the requirements phase of a development project. The outcomes of these requirements activities (identification of users, goals, environments, usability measures) provide direction for the design phase and basis for planning evaluations.

In practice, Nielsen’s attributes as such are too ambiguous to be used in determining the usability requirements.

My Comments: This part of the article, particularly on User-Centred Design, is good on the process of applying usability throughout the entire design and development process.

Understand and specify the context of use
The standard describes the activity ‘Understand and
specify the context of use’ as follows:
The characteristics of the users, tasks and the organizational and physical environment define the context in which the system is used. It is important to understand and identify the details of this context in order to guide early design decisions, and to provide a basis for evaluation.
Information should be gathered about the context of use of new products and systems. If an existing system is upgraded or enhanced, this information may already be available but should be checked. If there are extensive results form user feedback, help desk reports and other data, these provide a basis fro prioritizing user requirements for system modifications and changes.
The context in which the system is to be used should be identified in terms of the following:
a) The characteristics of the intended users: relevant characteristics of the users can include knowledge, skill, experience, education, training, physical
attributes, habits, preferences and capabilities. If necessary, define the characteristics of different types of users, for example, with different levels of
experience of performing different roles (maintainers, installers, etc).
b) The tasks the users are to perform: the description should include the overall goals of the use of the system. The characteristics of tasks that can influence usability should be described, e.g. the frequency and the duration of performance”…. Tasks should not be described solely in terms of the functions or features.
c) The environment in which the users are to use the system: the environment includes the hardware, software and materials to be used. Their description
can be in terms of a set of products, one or more of which can be the focus of human-centred specification or evaluation, or it can be in terms of a set of
attributes or performance characteristics of the hardware, software and other materials. Relevant characteristics of the physical and social environment
should also be described. These can include relevant standards, attributes of the wider technical environment, the physical, ambient, legislative and the social and cultural environment.

The output from this activity should be a description of the relevant characteristics of users, tasks and environment, which identifies what aspects have an important impact on the system design. (See ISO 9241-11 for more information about the context of use and a sample report.)
The context of use description should
a) Specify the range of intended users, tasks and environments in sufficient detail to support design activity;
b) Be derived from suitable sources;
c) Be confirmed by users or if they are not available, by those representing their interests in the process;
d) Be adequately documented;
e) Be made available to the design team at appropriate times and in appropriate forms to support design activities.

Specify the user and organizational requirements
In most design processes, there is a major activity specifying the functional and other requirements for the product or system. For human-centred design, this activity should be extended to create an explicit statement of user and organizational requirements in relation to the context of use description. The following aspects should
be considered in order to identify relevant requirements:
a) Required performance of the new system against operational and financial objectives;
b) Relevant statutory or legislative requirements, including safety and health;
c) Co-operation and communication between users and other relevant parties;
d) The users’ jobs (including allocation of tasks, users’ well-being, and motivation);
e) Task performance;
f) Work design and organization;
g) Management of change, including training and personnel to be involved;
h) Feasibility of operation and maintenance;
i) The human-computer interface and workstation design.
User and organizational requirements should be derive and objectives set with appropriate trade-offs identified between the different requirements.
The specification of user and organizational requirements should:
a) Identify the range of relevant users and other personnel in the design;
b) Provide a clear statement of the human-centred design goals;
c) Set appropriate priorities for the different requirements;
d) Provide measurable criteria against which the emerging design can be tested;
e) Be confirmed by the users or those representing their interests in the process;
f) Include any statutory or legislative requirements;
g) Be adequately documented.

ISO 13407 does not address the general complexity and specific challenges related to systematic identification of different users, identification of the different goals that users may have; nor determination of measures (effectiveness, efficiency, satisfaction) of usability. The determination of environments of use is addressed in most detailed manner.

DISCUSSION
ISO 9241-11 and ISO 13407 are two important standards related to usability: the former one provides the definition of usability and the latter one guidance for designing usability. We carried out an interpretive analysis of ISO 13407 from the viewpoint of the standard definition of usability from ISO 9241-11. The results show that ISO 13407 provides only partly guidance for designing usability as presumed by the definition. Guidance for describing users and environments are provided but very limited guidance is provided for the descriptions of user goals and usabilty measures, and generally for the process of producing the various outcomes.

References that I may want to read further in future:

1. ANSI. Common Industry Format for Usability Test Reports., NCITS 354-2001, 2001.

3. Bevan, N., Claridge, N., Maguire, M. and Athousaki, M., Specifying and evaluating usability requirements using the Common Industry Format: Four case studies. in IFIP 17th World Computer Conference 2002 - TC 13 Stream on Usability: Gaining a Competitive Edge, (Montreal, Canada, 2002), Kluwer Academic Publishers, 133-148.

8. Hix and Hartson. Developing User Interfaces: Ensuring Usability Through Product & Process.
John Wiley & Sons, 1993.
9. ISO/IEC. 13407 Human-Centred Design Processes for Interactive Systems, ISO/IEC 13407: 1999 (E), 1999.

12. Jokela, T., Making User-Centred Design Common Sense: Striving for an Unambiguous
and Communicative UCD Process Model. in NordiCHI 2002, (Aarhus, Denmark, 2002), ACM, 19-26.
13. Jokela, T. and Iivari, N., Systematic Determination of Quantitative Usability Requirements. in to be published in the proceedings of HCI International 2003, (Crete, 2003).

15. Mayhew, D.J. The Usability Engineering Lifecycle. Morgan Kaufman, San Fancisco,
1999.
16. Nielsen, J. Usability Engineering. Academic Press, Inc., San Diego, 1993.
17. Rosson, M.B. and Carroll, J.M. Usability Engineering. Scenario-Based Development of Human-Computer Interaction. Morgan Kaufmann Publishers, 2002.