Data Driven Testing

September 16, 2019

To ensure the quality and effectiveness of a software product, software engineers adopt various testing methodologies that help them evaluate and verify various aspects of the product. Data Driven Testing is one such testing methodology that is executed by the team during the software development life cycle (SDLC). Data driven testing is a test automation framework that stores critical data in spreadsheet format.

Therefore, to elaborate more on the importance of this testing technique, here is a detailed discussion on data driven testing.

What is Data Driven Testing?

A widely used software testing technique, data driven testing (DDT) is performed to test computer software. It is a kind of testing methodology where testing of the software product is carried out with the use of the defined sets of the specific inputs along with the sets of their corresponding and expected outputs. As the testing is driven by the available sets of the inputs, hence this technique is known as "Data Driven Testing".

Generally, the use of the DDT may be seen in the automated testing where the test input and its expected output is being retrieved from the specific data source such as data pool, excel files, ODBC sources, csv files, etc.

Features of Data Driven Testing:

From ensuring wide coverage to allowing multiple executions of tests, there are various important features of data driven testing, few of which are mentioned below:

A specific testing approach, based on the sets of inputs and their corresponding and expected outputs.
Ensures wide coverage for the automation process.
Provides a specific source of data with the recorded and pre-defined test scripts used for driving the test automation activity, which can be updated with the inclusion of the additional sets of inputs and outputs, effortlessly.
In this type of testing the verification data values for the test input and output are separated from the test code.
It allows multiple executions of same tests with different test data.
Here, the test input and output values are read from data files and loaded into variables in the test script.
The setup and control of the test environment in this process do not have any hard coded values.

Why Data Driven Testing Is Needed?

In software testing life cycle (STLC), there can be various reasons to perform data driven testing, as it plays a significant role in creating a software product that adeptly fulfills the requirements of the client/customer. Therefore, mentioned below are some of these reasons for performing data driven testing:

Ensures test script execution: The necessity of the data driven testing may be seen during the execution of the automated scripts where the recorded and pre-defined test scripts are used, to verify and validate the working of the software.
To ensure full test coverage: DDT approach can also be used for running the automated test scripts so as to have wide coverage. It provides the simple technique of creating a data source usually in the table format, having sets of input data along with their corresponding output values. The data source is dynamic which eases the task of further addition of the inputs and their expected outputs into it.
For adding more input data: This approach benefits the tester to add more input data and desired outputs without any effort, dynamically to expand the coverage of the testing without interrupting the automation process.
It can also be used for applications that require fixed set of actions to be performed, but with multifarious permutation and combinations of the various parameters, which form the test cases.
When there is a change in a particular functionality of the product, data driven testing allows the team to just adjust the specific function script.
It helps reduce the redundancy and unnecessary duplication of test scripts, which further enhance the quality as well as the effectiveness of testing.

How Data Driven Testing Works?

Before we move on to understanding the various aspects of DDT, it is crucial that we understand its workings, which will offer us a better insight into its process. Hence, the basic working of the DDT consists of the following activities in a subsequent manner.

Fetching the input data from the data sources.
Entering the retrieved input data into the software application(form).
Comparison of the actual results with the expected output.
Testing progress further with another set of input data.

To get a comprehensive understanding of the workings of data driven testing, click here.

How To Create a Data Driven Test?

To ensure the accuracy of data driven testing process, it is vital for the testing team to provide correct input values and to create a data driven test script that evaluates the software accurately when executed by the team. Moreover, following steps should be taken to into account while creating data driven test.

Test script should be designed and created with constant test data.
The constant test data should be replaced with some variables.
Create multiple sets of test data in a data storage.
Finally, the values read from the data storage should be assigned to the variables.

Data Driven Testing Infographics:

Best Practices of Data Driven Testing:

During data driven testing, the team adopts various practices that improve the quality of testing and helps them get accurate and expected results. Some of these steps are:

During DDT the team should use accurate and realistic information.
The navigation of the test flow should be coded within the test script.
To drive dynamic assertion, it is important for the team to use accurate data.
Both negative and positive outcomes of testing should be tested by the team to ensure the accuracy of testing.
Repurpose the data driven functional test for security, performance, and functional testing.
Virtual APIs can be executed with meaningful data.
Whenever possible, separate test data from the test code.
Configure error handling.

Advantages of Data Driven Testing:

When a particular testing technique is executed by the team, it offers them a variety of advantages, which further increases the efficiency of the testing. Likewise, when executed, data driven testing too offers various advantages to the team, few of which are mentioned below:

Reduces the execution time.
Offers better test storage.
The number of test cases and test scripts used is reduced.
The actions and functions can also be reused in multiple tests.
Changes in the test script do not impact the test data or the functionality of the product.
It can be performed during any phase of software development.
Enables the team to organize test data and verification data in one file.
During regression-testing, it allows the team to test data in multiple data sets.
Improved use of system and human resources.
Offers reduced test case maintenance.
Promotes reusability and repeatability, as the same test cases can be used for the various test.

Disadvantages of Data Driven Testing:

Though the list of advantages offered by data driven testing is non-exhaustive, it is still important that we mention few of its disadvantages, which are an integral part of this process. Therefore, some of the disadvantages of DDT are:

It requires high level technical skills.
Data validation is required.
When testing large pool of data validation becomes times consuming.
It requires continuous maintenance as DDT requires extensive coding.
The quality of testing is highly dependent on the automation skills of the implementing team.

Difference Between Data Driven Testing and Keyword Driven Testing:

Data Driven Testing	Keyword Driven Testing
1.This is an automation framework wherein the team read data files to get test input & output, which are added to the variables in the test script.	1. Keyword driven testing is a software testing methodology that used to define test cases based on the predefined keywords.
2. Data driven testing is also known as DDT.	2. Also known as Table Driven Testing or Action Word Based Testing.
3. The test cases prepared for DDT are designed with less code, which increases its simplicity and understandability.	3. Keyword driven testing can be applied to both manual & automation testing.
4. It requires high level programming & technical skills.	4. It can be performed without programming knowledge.
5. Data driven testing can be performed with the assistance of tools like SoapUI Pro and more.	5. Uses keyword, action words, or database tables to define the functionality to be tested.
6. Redundancy & unnecessary duplication of test scripts is reduced in DDT.	6. The tests cases created for keyword driven testing are concise, understandable, and easy to modify.
7. In DDT the input data can be stored in various data sources like XMC, CSV, databases, etc.	7. Developing keywords & its related functionalities is a time consuming process.
8. It can be performed during any phase of software development life cycle (SDLC).	8. One can perform automation testing in the earlier stages of development life cycle.
9. It is a scripting technique that stores test input and expected results in a table or spreadsheet.	9. Simplifies the maintenance of the changes in the software under test (SUT).

Key Points:

An integral part of software development life cycle (SDLC), data driven testing is performed by to ensure an optimum experience for the users. It allows them to store test input and output in a spreadsheet format, so that a single control script can execute all of the tests in the database table. Hence, DDT plays a critical role in validating the quality and effectiveness of the product and should be a part of a successful development process.