There are times when software contains very large number of test cases and when it seems that there is no end to testing for software, then there is something called schedule which is already defined to deliver the software with maximum quality and with all the bugs found during testing, fixed. One can’t do endless testing i.e. complete testing to deliver a quality product and also can’t guarantee that software is bug-free.
So, it is quite important to define the criteria to say that complete testing is done that we commonly call it exit criteria in testing language and project be delivered. The challenge with the exit criteria is that people in the project tend to justify the release of items for delivery before they are actually ready for delivery. The poor quality of project delivered leads to the unhappy customer at the end. So, it is quite important to define criteria for delivery of the quality project to the client.
What generally happens in the organization is that people in higher position want to reduce the criteria set for delivery in order to meet a delivery deadline, which is not liked by tester but are forced to do so by more influential people at higher positions.
Issues leading to Ineffective Complete Testing Status
- Believing that customer won’t mind a little inconvenience.
- Meeting the deadline even at the cost of compromised quality is more important for them.
- Wrong estimation of timeline defined for delivery.
- The hope of solving issues post-release.
- Stakeholders demand delivery regardless of the quality of the product.
Any modification in set criteria brings high risk to organization and customers and stakeholders as well. Though releasing schedule is an important thing to be considered for release but it should not be the only criterion. So, adherence to exit criteria is very important.
When say Complete Testing or Exit Criteria is Met
- 100% of the planned test has been completed as per test plan.
- No critical defects are there in the release.
- All the possible risk areas tested adequately.
- Regression is done properly for all cycles and no severe defect is left in a release.
Apart from this, there are some practical scenarios when complete testing is stopped and are supposed to be completed. Some of them are as below:
When say Enough Testing has been Done
- When the cost of fixing the issue is more than its impact on the software application.
- When a manager feels that sufficient | complete testing has been done and says to stop it.
- When the rate of detecting bug is very low.
- The product is stable and behaving as expected.
- When functional and regression test cases have been executed successfully.
Before defining the criteria for when say complete testing is done or what we say exit criteria, we define criteria for when to start testing or what is called Entry criteria in testing language.
Software Testing Entry Criteria
- 100% code coverage is ensured.
- Test cases are defined for each requirement. It is ensured that no requirement is there for which test case is not prepared.
- What happens in real time environment is that when there is release of application for testing after development is done, tester look at the basic criteria for testing in test plan and if they find that all entry criteria is met, they start testing with all defined processes and complete testing of the application, after that verify that whether the application meets all the exit criteria defined for deciding when complete testing is done. If tester finds that all the criteria are met, then he/she releases the application with release note to be deployed in the production environment, thus ensuring the quality product to the client.
During test planning, we define entry and exit criteria and is mentioned in the test plan. Rules of the game are set in this stage only. Much depends on the quality of criteria defined for Entry and Exit criteria in the test plan. So, it is an important part of test plan
In short, when to stop testing is defined in test planning stage. A major focus is on that all identified test cases have been executed successfully and there is no major bug or issue is left in the release. The schedule is also one of the main focus while defining the exit criteria. In order to meet tight deadlines for the release, sometimes, there is relaxation in defined criteria and testers need to prioritize test cases and ensure that no critical test cases with high business value to the customer is left, so as to deliver the quality product in defined timeline and do the best testing within defined timeline. Ideally, complete testing can be considered only if exit criteria are met, though sometimes, there is little relaxation in exit criteria made to achieve deadline or sometimes time is extended to meet exit criteria.
