Sep 19 - Nielsen, High-Cost Usability Sometimes Makes Sense (Alertbox)

High-Cost Usability Sometimes Makes Sense

Summary: Computing the net present value (NPV) lets you estimate the most profitable level of usability investment. For big projects, expensive usability can pay off.

I typically advocate "discount usability engineering" — that is, cheap and fast methods to immediately improve your user interface. But in some cases, it makes sense to invest more to get more.

When assessing your usability options, you'll often find two alternatives with drastically different costs, such as:
* Testing solely with domestic users vs. including users from one or more foreign countries to assess international usability issues
* Lab-based user testing vs. field studies at customer locations
* Testing only your own design vs. including 2-3 competitors' designs ($28K vs. $45K on our price list)
* Paper prototyping vs. a more fully implemented prototype
* Hiring usability professionals at different skill levels (and thus different salaries: $55K vs. $80K for entry-level vs. somewhat experienced staff, respectively)

To decide which option to choose, you have to combine 4 parameters:
* The project's expected value
* The cost of the usability alternatives
* The estimated difference in outcome for the alternatives
* The rate at which you discount future money streams to account for uncertainty

So, we basically have 3 options: no usability, cheap usability, or expensive usability.

No usability is the easiest to compute, since we don't have to consider the cost of the usability alternatives. We assumed a value of $1M and a discount rate of 100%, so we can easily compute the project's present value: $500K before deducting project costs. So, if design and development (other than usability) cost, say, $200K, the project has an NPV (net present value) of $300K, and should go ahead.

Cheap usability adds 90% to the project's value, resulting in $1.9M, or $950K after applying the discount rate. We then deduct the $10K usability cost, resulting in a present value of $940K. In other words, usability has added $440K to our project's expected present value. After deducting the $200K development costs, the project now has an overall NPV of $740K.

Expensive usability adds 110% to the value, resulting in $2.1M, or $1,050,000 after applying the discount rate. We now have to deduct $40K, resulting in a present value of $1,010,000. Thus, usability has added $510K to the expected present value. After deducting the development costs, overall NPV is $810K.

Under our assumptions, using expensive rather than cheap usability increased NPV by $70K, so we should go ahead and use the expensive usability approach.
These calculations assume that development costs are the same with and without usability.

Heuristics for Choosing: Cheap vs. Expensive

If you'd rather avoid the fancy calculations, following are five general rules for choosing between cheap and expensive usability.

1. The higher the project's expected value, the more you should invest in usability.
If your project's value is low, you won't earn much by doubling it.
For example, if your project has an expected value of $5K, you shouldn't even do cheap usability, because gaining an additional $5K isn't enough to justify spending $10K. (However, there are still some ultra-cheap usability methods you can use for tiny projects.)

2. The higher your project uncertainty, the higher your discount rate.
In such cases, cheap usability options are typically the way to go. Yes, the more expensive usability option would offer higher gains if you were ultimately successful, but your probability of a total loss would be too great to justify the investment.

3. The earlier in your project you employ usability, the more you should invest because you can implement the added findings without much rework. Late studies should be cheap; you won't be able to make many fundamental changes anyway.
Note the tension between project stage and uncertainty: early in the project, your uncertainty is higher (which counts against big expenses), but your potential for implementing the gains is bigger (which counts for thorough studies).

4. The bigger the expected difference in outcomes, the more you should go with expensive methods.
When the expected outcome difference is small, save your money and go with the cheap methods. Quantitative studies are a great example: they're very expensive and usually only worth the cost for huge projects because they otherwise add relatively little value. A similar analysis can be made for international studies: if you have few overseas customers, the added cost of testing them won't be worth the expense.

5. The more expensive the high-cost methods are, the less you should use them.
In our example, the NPVs will be identical if the expensive method costs $110K. Thus, if the expensive method is 20% better than the cheap method, you should use it as long as it's no more than 10 times as expensive as the cheap method. If it's 11 times as expensive, you won't get enough added benefits to justify the added cost.

Conclusion:
When it comes to selecting usability methods, there are many parameters to consider, and many different scenarios. That's why both expensive and cheap usability methods make sense under the appropriate circumstances.

source:
Jakob Nielsen's Alertbox, November 5, 2007:
High-Cost Usability Sometimes Makes Sense
http://www.useit.com/alertbox/expensive-usability.html
High-Cost Usability Sometimes Makes Sense (Jakob Nielsen's Alertbox)