Test Prioritization or Test Case Prioritization

Feb 25, 2020

Testing a software or application is the most important phase of Software Development Life Cycle (SDLC). It plays a very essential role in deciding the quality and functionality of the software and ensures that the end product is as per client’s demand. The Testing of the software is done to detect faults and errors in its functionality, on the basis of which the performance and quality of the software is further improved. Test Prioritization is an extension of software testing, which is implemented to decide the priority of the test cases.

This priority of the test cases is decided through various factors that depend upon the need of the software, which are then assigned to the test cases for further processes. By prioritizing test cases, testers and developers are able to minimize the time and cost of software testing phase and make sure that the delivered product is of exceptional quality.

What is Test Prioritization?

The term "test prioritization" refers to the subjective and difficult part of testing that allows testers to manage risks, plan tests, consider cost value, and be analytical about which test to run in the context of the specific project. This process, is well known as Test Case Prioritization, which is a method of prioritizing and scheduling test cases. This techniques is used in order to run test cases of higher priority in order to minimize time, cost and efforts during software testing phase.

Furthermore, test case prioritization provides assistance with regression testing and improves its performance. Through this method, testers can easily run test cases, which have the highest priority and provide earlier defect faults. Also, an improved rate of fault detection during the testing phase, allows faster feedback of the system that is under test. With test case prioritization software engineers can get assistance in correcting faults earlier than might otherwise be possible. Moreover, to decide the priority of test cases, various factors depending upon the need of the software are decided.

Aspects to Consider While Prioritizing Test Cases

Test case prioritization is an extremely important element of software testing. It allows testers to test highest priority test cases, which helps them resolve most critical issues earlier in the testing phase. However, it is essential for the testers to follow several factors while executing test case prioritization.

These factors ensures that proper testing is executed in appropriate order and also validates that all discrepancies are detected during the process of testing. The aspects that are required to be considered while prioritizing test cases are:

The tester should consider the usage frequency of a function or the probability of failure in software use.
The failure risks of the test cases should be measured.
Visibility of a failure for the end user is a further criteria for prioritization of the test cases.
Test cases can be chosen as per the priority of the requirements of all the stakeholders.
Quality characteristics may have different importance to the customer or client.
Prioritization can also be done from the perspective of the development or system architecture.
Areas with complex coding and with past histories of problems should be taken into account.

Benefits Of Test Case Prioritization:

From reducing the cost and time of testing, to helping testers in managing the testing procedure, test case prioritization is extremely beneficial in software testing phase. It is a process which takes into consideration that no extra efforts are wasted in the process of testing, as it prioritises the test cases that require earlier and extra attention and focus of the team. Other benefits of test prioritization are:

Improves the performance of the regression testing.
Arranges test cases in an order, to obtain maximum available fault that is detected in a shorter time.
In this, the priority is given to the test cases that are performed based on the multiple criteria and hybrid criteria, to enhance the effectiveness of the time and cost for proposed technique.
It also ensures that the most important test cases are executed early in the Software Development Life Cycle (SDLC).
Allows testers to find defects and errors in the system as soon as possible.

Test Case Prioritization Techniques :

The process of test case prioritization is a difficult one and it requires knowledge of expert and experienced team of testers. Moreover, there are several factors that affect a tester’s decision of choosing a particular prioritization technique. These not only affect the process of testing, but also aim at making the process time and cost effective. Hence, here are some of the major techniques of test case prioritization used by software engineers all around the world.

Prioritization Based on Average Percentage Fault Detected: This technique is based on the rate at which fault is detected in the code under test. In this technique two different criteria are used to decide a factor on the basis of which the priorities to the test cases are assigned.
Prioritization Using Faulty Severity: Here priority to the test cases are assigned depending upon the priority of the requirement to be tested. The requirement to be tested first is assigned the higher weight and the test cases covering that requirement are given higher priority of execution. Moreover, in this approach the requirements considered are based on fault severity, which is the number of times a fault can occur in the code. This code is then required to be tested after which it is given the higher weight. The some of the factors that decide the weight of the requirement are:

Business value measure.
Project change volatility.
Development complexity.
Fault proneness of requirement.

Prioritization in Case of Regression Testing: To achieve prioritization in the case of regression testing testers can choose from overall nine techniques. Each of these techniques various elements of the test cases are measured, which help in deciding their priority. These nine techniques are:

Random Prioritization: This is applied to have an additional control in studies where the test cases are ordered randomly in the test suite.
Optimal Prioritization: Known faults are used so that its results can be used to measure the effects of other prioritization techniques that will be used.
Total statement coverage prioritization: It instruments the program with any test cases and finds out which statements were covered by the test cases, which are then prioritized on the basis of the number of statements they covered.
Additional statement coverage prioritization: Covers the shortcomings of Total Statement coverage prioritization and iteratively selects test cases that give highest statement coverage, then it adjusts the coverage information on rest of the test cases to find out the statements not yet covered.
Total branch coverage prioritization: Mostly similar to additional statement coverage technique, it uses test coverage measured in form of program branches as there was the case of statements in other technique.
Additional branch coverage prioritization: This technique is similar to additional statement coverage technique, with the only difference that it uses test coverage measured in terms of program branches and not statements.
Total fault-exposing potential prioritization: As some test cases can expose faults more easily as compared to others, so it is called as fault-exposing potential of a test case.
Additional fault-exposing potential Prioritization: As additions were made in total coverage and branch coverage prioritization, extensions were made in total fault-exposing potential prioritization and as a result this technique is created, where total fault exposing potential is extended to create additional fault exposing potential prioritization.

Challenges in Test Prioritization:

While executing test case prioritization, the team of testers can encounter several challenges that make the process of testing time consuming and difficult. Therefore, the team of testers are required to pay additional attention to the following mentioned challenges. By taking care of these factors, the testers can initiate safe and secure testing as well as get accurate result. These challenges are:

Detecting change and affected parts of the program.
Scalability to handle complex systems.
Integrating seamlessly into development process.

Conclusion:

Test case prioritization is best technique to ensure the effectiveness as well as the quality of a product, during its development and testing process. It is one such method that prioritizes and schedules test cases according their highest and lowest requirement. Through test case prioritization, testers can effectively make sure that the most important test cases are executed first.

This way important problems and defects can be found in the software or application as early as possible. Moreover, this testing provides assistance in regression testing and allows testers to order their test cases, so that test cases with highest priority are executed in the testing process, than the lower priority test cases. In short, this subjective and difficult part of testing is about risk management, test planning, and being analytical about the tests that are required to be executed in the context of the project that is specified.