Software testing and risk reduction

Joel Spolsky describes five different worlds of software development, each with their own problems, and describes how we should consider which world we are operating in when deciding how to go about developing something.

Often, people move from one world to another, from e.g. shrinkwrap development to internal applications, but still think the approach they initially learned is the correct one. This is unlikely to be the case. An application used by a small number of people dealing with medical data will need to be developed in a different way to a program used by many thousands of people to design their gardens.

The former will need to ensure that data does not become corrupt, and is kept securely. The latter will need to run on a wide variety of operating systems and be easy and intuitive to use. The two different types of applications call for different testing methodologies.

When testing, you should consider what the risks are in your application, and then:

• How many people might be affected if it goes wrong


• What is the chance this will go wrong


• What is the cost per person if it does go wrong

You can then multiply the 3 figures together, and work out a strategy to reduce the significant risks. It may be that this is to use test driven development when developing the software. It may be that it is to invest in usability labs, to ensure people find your application easy to use. It may be that it's best to test on a wide range of different hardware configurations.

If none of the risks are significant, as with throwaway applications, you need to do very little. If all of them are, you should do a lot more testing.