Sep 11 - Usability Requirements Specification (CISU-R part 3)

6. Usability requirements specification

6.1 General

Usability requirements based on the CISU-R have three parts: the context of use, the performance and satisfaction criteria, and the test method. All usability requirements include all three parts, but there are three levels of compliance.

The context of use: a description of the intended users, their goals, associated equipment (including hardware, software, and materials), the physical and social environment in which the product will be used, and examples of scenarios of use. (see Clause 6.1)
- A complete description of the context of use as specified in Section 6.2 shall be included in all usability requirements (Levels 1, 2, and 3). Additional detail may be added during the design and development process.

Performance and satisfaction criteria: ways in which the usability of the product can be measured, specified for the main scenarios of use. They may include target values, or may specify how these values will be identified (see Clause 6.3). These criteria can be specified to three levels of completeness and precision:
- Level 1: the requirements shall identify the criteria appropriate for successful use of the product, and the relative importance of each and identifying those that are practical to measure directly (see Clause 6.3.1).
- Level 2: the requirements shall specify target values for the criteria, or a range of acceptable values, (see Clause 6.3.2).
- Level 3: the requirements shall include specific values for measures of user performance and satisfaction to be tested, or a method of determining those values (see Clause 6.3.3).

The test method: how the product will be tested to determine whether the usability requirements have been met. The specification of the test method may include only an identification of the appropriate test methods, may describe a method and identify the test context, or may include a complete specification of a test method (see Clause 6.4). These criteria can be specified to three levels or detail:
- Level 1: the requirements shall identify the types of methods that are appropriate for the product and criteria selected (see Clause 6.4.1).
- Level 2: the requirements shall include a preliminary test method and the criteria to be evaluated (see Clause 6.4.2).
- Level 3: the requirements shall include a complete test protocol for a user performance test, as specified in Annex C (see Clause 6.4.3).

These levels are intended to ensure that usability requirements can be developed with the CISU-R for all types of projects, from the smallest or most informal to the most complex or formally specified products.
The three levels of compliance allow usability requirements to be written to an appropriate level of completeness and precision for the product.
At the same time, they encourage all projects to reach at least Level 1 compliance.

6.2 Context of use

6.2.1. General

For all levels of compliance with this standard, the context of use shall include descriptions of:
- Stakeholders
- User groups
- Goals and tasks
- Technical environment (equipment)
- Physical and social environments
- Scenarios of use for the most important goals

6.2.2. Stakeholders

Usability requirements shall include a description of all groups of people who are identified as stakeholders, including:
- A list of all groups who have a legitimate interest in the product throughout its life cycle.
- Their roles and interests in the product. EXAMPLE 1. Role: marketing product manager; interest: business needs solutions EXAMPLE 2 Role: user/administrative assistant; interest facilitation of administrative duties
- If any of the stakeholders will not be considered in identifying requirements, the rationale for excluding them.

6.2.3. User groups

Usability requirements shall include a list of all anticipated user groups.
Requirements should be provided for the user groups who use a product most frequently or who are critical for business purposes. Key characteristics and capabilities of these primary user groups shall be defined.

6.2.4. Goals

The main goals for each user group shall be listed, without reference to any specific means of achieving them. The goals should be an intended outcome of value to the user or business, for example: accurately completing a particular form, locating the most relevant information, or successfully setting up a computer.

6.2.5. Technical environment (equipment)

Any relevant aspects of the intended or anticipated computing or other technical environment shall be specified.
Examples (particularly applicable to software products) include:
1 Hardware configuration: e.g., processor speed, memory size, network, storage, input, and output devices.
2 Screen type (CRT or LCD), resolution, and color depth. If relevant, also include display (monitor) size and whether multiple monitors are required.
3 Print media size and print resolution.
4 Whether visual interface elements (such as text) can vary in size and the size(s) available.
5 Software configuration: e.g., browser version, operating system version, middleware, database. 6 Assistive technologies available to users with disabilities.
7 Documentation and support materials.

6.2.6. Physical and social environments

Any aspects of the expected physical and social environments that can influence usability shall be specified:
1 Physical environment in which the product will be used, including location and relevant physical conditions, such as temperature or lighting.
2 Organizational environment, including group work dynamics, time pressures, supervision, and support.
3 Any physical, health or safety issues.
4 Any possible financial or security risks

6.2.7. Scenarios of use

The goals and tasks that will form the basis of the requirements shall be illustrated with scenarios of use. These scenarios describe how users meet their goals. They are examples of users’ activities, motivations, and how they carry out their tasks, using the product in a specific situation.

6.2.8. Training

If users are expected to study any documentation, training materials or courses before using the product, the following information shall be specified:
1 Which user groups are expected to study the materials.
2 What previous knowledge or experience is required.
3 Which goals and tasks are included in the training materials.

6.3 Usability performance and satisfaction criteria

6.3.1. Identifying relevant usability criteria (Level 1)

Performance and satisfaction criteria identify the measures for the usability of a product. The first level in specifying performance and satisfaction criteria is to identify the criteria relevant to the context of use for the product.

The following information shall be provided for Level 1 compliance:

a) The types of performance and satisfaction criteria appropriate for successful use of the product.
−These might include performance criteria such as task completion rate or time on task, subjective “ease of use” scores, indirect measures such as number of steps in a task, or a list of features to support usability that can be evaluated with a checklist.
−Different user groups or scenarios may have different criteria.

b) The relative importance of each criteria to the success of the product.

Examples of performance and satisfaction criteria include: the unassisted completion rate (effectiveness), the mean time taken to successfully complete each goal (efficiency), satisfaction scores on a specific questionnaire, efficiency relative to a past benchmark or other product. Annex B provides additional information about selecting appropriate criteria and best practices for how they should be measured.

6.3.2. Defining target values for criteria (Level 2)

The second step in specifying performance and satisfaction criteria is to provide target values for the criteria, or indicate a method for determining those values later in the project.

Target values are the minimum acceptable value, or a range of acceptable values, for the criteria. They may be based on the value for an existing or competitor system used for the same tasks and goals, based on expert judgment or specified by stakeholders. Additional target values better than the minimum acceptable value may also be specified.
For Level 2 compliance, the usability criteria shall include target values or a range of acceptable values for these criteria. Target values may be in actual numbers, percentages, average or means, a range of values, or a scale. they may be absolute, or relative to performance benchmarks

Each target value may be given as:
a definite requirement, or
a provisional requirement subject to further negotiation, or
an objective for guidance.

For effectiveness, efficiency, and satisfaction criteria, either target values shall be given, or a note provided explaining that either:
a target value will be provided later, or
why a target value cannot be established (for example for a completely new product), or
why no target value is needed (for example the measure is of low importance to the user or organization).

6.3.3. Defining specific criterion values (Level 3)

For Level 3 compliance the requirements shall include specific values for each criterion. These may be validated by benchmark usability testing, business requirements, or other methods.

6.3.4. Including use of learning materials in the criteria

If any documentation, training materials or course will be studied before the product is used, criteria for the usability of the training materials in at least one training scenario (effectiveness, efficiency, and satisfaction when completing training goals) should be included.
EXAMPLE 1 Given 2 hours of training with the online training program, a user will be able to complete the item identification task under 1 minute and with fewer than 2 errors.
EXAMPLE 2 After reading the owner’s manual a user will be able to access and listen to a cellphone message on the first attempt.

6.4 Methods for testing

6.4.1. Identifying evaluation methods (Level 1)

The final part of a usability requirement identifies methods to evaluate whether the criteria have been met and the context in which the criteria will be measured.

The first level in specifying test methods is to identify methods that are appropriate for the product. For some products these methods may include checklists or other inspection methods, or indirect measures such as business metrics, but ideally, they include methods in which people who are representative of the user groups test the product.
For Level 1 compliance, the usability requirements shall include a list of usability methods that can be used to determine whether the usability requirements have been met.

6.4.2. Specifying a preliminary usability evaluation method (Level 2)

The second level in specifying test methods is to describe the method to be used to test whether the usability requirements have been met, and the context in which the measurements will be made. This level adds enough information to describe the basic test method to a usability professional, but is not a complete specification of the method.

For Level 2 compliance, the preliminary evaluation method shall include a description of the methods used to measure the product on all of the criteria.

If a usability test method is specified, the description shall include:
1 Goals of the test, including task scenarios to be tested and criteria for successful completion of each goal (see Annex C.2.1.a and b).
2 The user groups (identified in the context of use) to be included in the test (see Annex C.1.b).
3 The type of test facility, including the setting and type of space, and how relevant aspects of the intended environment will be simulated (see Annex C2.2).
4 Computing environment, which may be specific or anticipated (for example, specifying the "current version at the time of the test" (see C.2.3).
5 A general description of the test procedure (see C.3.1).

My Comments: For my PhD research, one of my deliverables is procedure of usability evaluation using UET. I will follow this Standard for description of usability test/evaluation method.

6.4.3. Specifying a complete usability test method (Level 3)

For level 3 compliance, a full protocol for a usability test shall be specified, providing the information required in Annex C.

The specification in Annex C is consistent with ISO/IEC 25062 (Common Industry Format (CIF) for usability test reports), which should be used for reporting the results. ISO/IEC 25062 requires reporting on the results of measurements of efficiency, effectiveness and satisfaction, or an explanation of why any of these metrics are not meaningful for this product.

6.5 Traceability

The following information should be documented to assist the development organization in maintaining and refining the requirements:
1 The source of the requirements.
2 How the usability requirements support the stakeholders’ business objectives for the product.
3 Why the goals were selected.
4 The source of each goal.
5 Who requested this requirement.
6 Whether the requirement is essential or desirable.

6.6 Document control

The following information should be included to track different versions of the requirements document:
1 Version number
2 Date of change
3 Nature of change
4 Person making the change
5 Person approving the change.

Source:
NISTIR 7432
Common Industry Specification for Usability - Requirements
June 2007
NIST
National Institute of Standards and Technology
Technology Administration, U.S. Department of Commerce