All About Testing Data Warehousing Applications

March 1, 2019

---

In the 21^st century, Business enterprises the world over, are finding it difficult to ignore the utility of e-commerce from their long term growth plans. E-commerce owes a lot of its success, to the readily available depositories of data, relating to sales and marketing. These Depositories of information are called Data warehouses. As more and more business houses become heavily reliant on data for their strategic decision making, the area of data warehouse testing naturally assumes importance. Here in this blog, we throw light on what constitutes warehouse testing, the nuances of this form of testing and the challenges involved...

Working strategy for data warehouse testing:

Any strategy for testing data warehouses will involve an approach revolving around its chief structures which are:

• Sources: these are the operational systems such as ERP which generate the sales and marketing reports.
• ETL or the Extract, Transform and Load applications
• Data marts: The small sub sets of a data warehouse, the data marts draw upon the data from the resources and focus on a single functional area. They are pretty fast and easier to implement.

Some techniques of Data warehouse testing:

1. Test for completeness of data:

   This covers validation of all the records and fields and to see to it that every field has been loaded. A holistic comparison of the data sourced, loaded, as also that, which has been rejected, has been found to be quite useful for this examination.

2. Data transformation test:

   This involves the correct conversion of data according to the business logic. There is flexibility of manual/automated approach to testing. However the manual tester employed would need to have an expert understanding of the ETL logic. An alternative to this, is the automatic data profiling and movement validation.

3. Quality Testing:

   Many a times, users forget to add a number to a data requiring alpha numeric characters and are notified for correction of the same. This testing covers precisely this aspect. i.e. data rejection, notification and its subsequent rectification. This also includes scenarios for duplicity in records sourced, or null key values.

4. Performance testing for data warehouses:

   As the use of e-commerce is synonymous with humongous amount of reports and user feedbacks. With a substantial increase in load, the performance of queries asked, may suffer a downgrade in quality. Such a situation calls for a strong and robust technical architecture. An architecture, the design of which, is bereft of inconsistencies, such as multiple reading of reports or pointless intermediate pages.

5. Regression testing:

   With the release of new variants/upgrades, it becomes important to have a re-evaluation of the basic functions of each system of the data warehouse. At the time of creation of test case it is essential to keep in mind, it's utility for multiple times as the release of new upgrades is a regular occurrence due to detection of bug fixes and repairs.

6. User acceptability test:

   The participation of the business stakeholders (who form the user base) in this test is vital to the success of data warehouse. The sample data used, is in close resemblance to the information related to actual production. The users most often have curiosity for the sources and the process of extraction involved for data mining.

In today's day and age, correct information reflecting the major trends and covering careful analysis of the product performance is worth its weight in gold. An organization which rightfully invests, in setting up of robust and meticulous testing setup for data warehouse will always reap rich rewards by staying ahead of the competition curve.