When Should One Stop Testing

July 19, 2016

---

The life of testing is not finite just as counting is not finite. It has to be applied when there's a need. Actually there can be various dimensions to this fact. We can think of testing to be complete when a tester full fills the requirements by running complete set of test cases or its the intuitiveness of the tester who feels that the testing has covered almost the entire application. When complexities aren't big enough the testing process may not take a very long route.

Lets try to analyse the different approaches that can be applied to judge when to stop testing.

• Test Coverage :To what extent and how elaborately are we able to extend the span of testing can be measured in terms of the platforms it is being tested. For instance, we are developing a web application, we need to check on mobile and other devices on various browser platforms. It is quite likely that the application may not perform well under all the circumstances alike. Hence it is no guarantee that testing complete, because amendments are going to take place till the application is alive.
• Defect Patterns -Defects often occur in a pattern. There may be a situation where testers may encounter defect in one device/platform but the same works fine in other device. This will surely not be a good time to sum up the testing process.
• Show stoppers -Sometimes the software project may give rise to such problems that may bring the entire process back to the discussion zone. The team may need to find out few critical errors, which needs an in depth analysis.

Bugs are hard to deal with. Even after a software application is deployed the end users may report of some bug/defect which may have been overlooked by the testers. It is therefore a good practice to proceed with the testing process maintaining a check list along. However, they say there's always a solution to every problem. We can devise a strategy to deal with the question of when to stop testing.

Ways to tackle testing :

• Decision based upon number of passed or failed test cases- One can always prepare a predefined set of test cases before the testing cycle, executing every test case in every testing cycle, and finally if all the test cases pass end testing.
• Decision based upon metrics - Adopting various mathematical calculations to get a calculation based result. Defect measurement can be done on the basis of mean time between failure(MTBF), coverage metrics, by calculating defect density.

Conclusion :

There are certain situations when time and budget pose a constraint, we have almost arrived at the deadline date of project completion or there may be few bugs which are of high priority therefore they are catered first. There is no definite scenario where one can say that testing is complete. It is a a situation based decision where one can just randomly decide to stop the testing or reach a consensus in a scenario when coverage of code is good, size of defects and their impact is low, number of high severity bugs are low. There is no defined rule which can be applied as to when should one stop testing, it is the ingenuity of a tester to be able to come to a concensus. It is a matter of experience and application of knowledge.