Nov 4 - Rubin, Setting Up A Testing Environment

Chapter 3
Setting Up A Testing Environment


SIMPLE SINGLE-ROOM SETUP

Advantages
1. The test monitor has an excellent sense of what is going on with the participant.
2. During early phase exploratory tests, where much interaction is desired to interrogate the participant during the test, this position accentuates a sense of teamwork.
For difficult tests, where the participant has to struggle with the material, it enables you to encourage and overcome the participant's self-consciousness.

Disadvantages
1. The test monitor's behavior can affect the behavior of the participant.
2. There is very limited space for observers.

MODIFIED SINGLE-ROOM SETUP

Advantages
1. The test monitor is more free to move about, take notes, use data logging software while test is going on, and yet is still within visual proximity of the participant.
2, The participant does not have a complete sense of isolation, since the participant is still in the room with the test monitor.
3. This setup is more likely to encourage the participant to think aloud, than if left alone in the test room.

Disadvantages
1. Loss of proximity to the participant limits what the test monitor can see directly of the proceedings.
2. If the test monitor is directly behind the participant and within ten feet or so, the test monitor would make the participant feel very uneasy and overly self-conscious, since the test monitor is not within the participant's peripheral vision.
3. As with simple single-room setup, there is limited space for observers.

ELECTRONIC OBSERVATION ROOM SETUP

Advantages
1. All the advantages of the simple single-roon setup also accure to this setup.
2. Observers get to view the test as much as they like without having to worry about interferring.

Disadvantages
1. As with the simple single-room setup, the test monitor's behavior can adversely affect the test.
2. Unless you have a permanent setup, you will need to tie up two conference rooms for up to a week.

CLASSIC USABILITY LABORATORY SETUP

Advantages
1. It allows unparalleled unobstrusive data collection.
2. Being a soundproof room, the testing staff and observers can easily communicate among themselves about the proceedings and discuss possible product improvements.
3. This environment can accomodate many observers viewing the test at one time.

Disadvantages
1. Depending on the skill of the testing staff, the control room setup can create a very impersonal environment.
2. Unless you have ideal camera placements, you may not be able to see exactly what the participant is viewing or experiencing.
3. For exploratory tests, the control room setup may offer litter advantage to monitoring the test, since you will need to be next to the participant in any case.

MOBILE, OR TRAVELING, LAB

Advantages
1. This is a very cost-effective solution since no costs are incurred for a physical location.
2. Since the equipment is portable, you are free to rent space near existing customers, such as a hotel conference room or a company sales office.
3. Storing all equipment in a locakable cart decreases the probabilty of equipment becoming damaged or misplaced and makes it very easy to "set up" the test site in very little time.

Disadvantages
1. Not having a permanent space means having to settle for whatever space is available.
2. If you will be doing a lot of testing, moving around delicate equipment can play havoc on that equipment.


Source:
Jeffrey RUBIN. Handbook of Usability Testing: How To Plan, Design and Conduct Effective Tests. John Wiley & Sons, New York, 1994.