During the process of software testing, preparing documents and records is usually overlooked by software testers. Documentation is treated as an insignificant activity, which does not require any consideration and attention of the team. However, these “insignificant” documents are extremely valuable and should be treated with immense care and accuracy.
It is usually noticed that software projects with proper and careful documentation are more mature and reliable than the once with no documentation at all. Furthermore, these documents also affect the quality of the software and make the process of software testing and development easy and convenient. Though time consuming, creating these documents often offers several benefits to software engineers and the organization they are developed for, as they help them save time, money, and efforts invested in the process.
To establish the importance of these documents IEEE has established a standard- IEEE 829-1998- which further makes preparing these documents easy as well as mandatory. Therefore, to define the importance of software test documents and IEEE standard 829-1998, following is a detailed discussion on the same.
Defining IEEE 829-1998:
Published by Institute of Electrical & Electronics Engineers, IEEE std 829-1998 is a software & system test document that helps software testers record the process of software testing, with great care and accuracy. This standard, which is also known as Standard for Software Test Document and System Test Document specifies the format of the documents that are developed during the process of software testing. The documents that are covered by this standard are:
Test Plan:
Developed during the early stages of Software Development Life Cycle (SDLC), the importance of test plan document is monumental during the process of testing. Used by testers to formally conduct software testing, test plan is a document that details a systematic approach to test the software system or an application. This plan contains a detailed understanding of the eventual work flow and clearly defines the activities performed by software testers during the process of testing.
Additionally, the it states the scope and objective of testing and helps software testers get accurate and expected result. In short, a test plan acts as a guide that ensures success of software testing and helps control risks as well as defects. According to IEEE std 829, test plans include the following information and is presented in the same format.
Test Plan Identifier: A significant part of the test plan document, test plan identifier is used by companies and organizations to create a unique identity for their test plan. Usually generated by the team manager or lead, this identifier helps them identify the level of test plan as well as the level of the software it is related to. With the assistance of this plan software testers are able to differentiate between the various types of plans such as master plan, level plan, an integrated plan, among others.
Introduction: This part of the document outlines what will be tested during the process of software testing. Here, the brief introduction is provided by the team regarding the project plan, quality assurance plan, configuration management plan, standards, etc. The team identifies the scope of plan in relation to the software project it is related to. The various information provided in this document regarding the software testing process are:
Resources & budget constraints.
Scope of testing efforts.
Process used for change control.
Details about communication & coordination of key activities.
Test Items: Here the various test artifacts, test cases, test scripts, and other aspects related to the software are included, which are tested by the team during the process of software testing. Other important items included in this list are:
Requirements specifications.
User guides.
Design Specification.
Installation manuals & guides, etc. .
Features to be Tested: Narrates the items and components that require testing from the users point. This section of the test plan includes the design specification and other requirements suggested by the client.
Test Approach: Another important part of the document, here the details about software testing techniques as well as its strategy is included along with the test plan, test types, and methodologies.
Item Pass/Fail Criteria: At this stage of the test plan a criteria is specified, which is used by the team to determine whether a test item or test case has passed the testing or not.
Suspension Criteria & Resumption Requirement: Similar to the item pass and fail criteria, the team defines and specifies the criteria based on which testing was suspended. Additionally, they highlight the testing activities that will be performed once the testing is resumed.
Test Deliverables: This includes all the necessary information and documents that will be delivered to the client after the culmination of the testing activities, such as :
Test Defect Report.
Release notes.
Installation & configuration document.
Test Plan.
Test incident report.
Test summary report, etc.
Testing Tasks: To offer a proper insight into the testing activities, the team defines the various testing tasks performed by them to ensure the quality of the product. These include detecting, tracking, analyzing and resolving defects, bugs, and any other discrepancies. Moreover, there should be specific tasks defined for each deliverable.
Environmental Needs: To ensure the quality of the software it is important for the team to consider the test environment requirements, such as the hardware, software, network, among other things before the inception of testing. Moreover, any related tool or resources should also be considered by the team here.
Responsibilities: Test plan also includes a brief about the responsibilities and roles assigned to various members of the team, the testers, etc.
Staffing & Training Needs: The skills and knowledge of the testers is defined, along with any important training that was given to them during the process of software development. The total number of individuals involved in the procedure is also mentioned.
Schedule: Finally the schedule of software testing and their milestones are mentioned in the document. Moreover, to validate that these milestones were achieved within the specified time period, related resources and documents are provided. Also, this schedule should be realistic and should conform to estimations.
Risks & Contingencies: Before stating the approvals of the various steps and actions, the team identifies all the risks present in the software and prepare the mitigation and contingency plan accordingly.
Approvals: At the end of the document roles and designation of the people responsible for the approval of the document and process is provided by the team, which is later approved via signatures from these authorities.
Test Design Specification:
Test design specification is another important document that is usually missing from the various deliverables offered to the client and the customer during product release. It offers an insight into the steps taken to design the test cases for the process of software testing. This document specifies the test conditions of various test items and identifies the associated high level test cases. The purpose of test design specification is to define the refinements of the test approach presented in the test plan.
Test Conditions: It defines the variety of specifications and requirements that ought to be followed by the testers to ensure the quality of the software application. It can either be a piece of functionality or anything that needs to be verified before the release of the product. In short, this covers the various coverage items & specifications of an application.
Detailed Approach: Here, all the approaches used for software execution are described by the testers to eradicate any misunderstanding, confusion and uncertainties among the team and other stakeholders of the project.
High Level Test Cases: It defines the functionality of the software without covering its functionality in detail. They cover the application in a broader way and offers the flexibility to explore other important test cases.
Test Incident Report:
Usually confused with the test defect or summary report, test incident report is a document prepared by the testers after the conclusion of software testing activities. Test incident report keeps a track of all the incidents found during the software testing period, which impact the performance and functionality of the software. Moreover, these incidents cause the software to behave in a questionable manner, which further deviates its output/results from the expected results.
Report ID: A unique identity in the form of a number is assigned to the document by the organization to help differentiate it from other test incident reports and documents.
Summary: In order to help others understand the various incidents found during the process of testing and the steps taken to resolve them, a detailed summary is offered by the team of tester. It comprises of assortment of information about the whole process with elaborate resources and evidence.
Description: Unlike the summary, the focus here is on defining the incident with as much details as possible. Moreover, any information that was not included in the summary is also mentioned here.
Impact: Finally, the impact of these incidents on the whole software or application is defined. The actual or potential damage, its severity or priority, and the steps taken to resolve it, every crucial detail about the incident is offered here to help the team understand its impact on the software.
Test Summary Report:
The significance of test summary report is monumental. It offers a summary of the entire software testing life cycle, with details about the heterogeneous activities performed during the process. Prepared in a comprehensive and decipherable manner, this document offers the client and other stakeholders an insight into the testing activities as well as an assessment of the steps taken to achieve the expected results. Hence, the format for test summary report defined by IEEE std 829-1998 is:
Summary: A summary of the whole report is recorded here with suitable resources and material, which assists the reader determining the version and release of the project that is being reported by them. Moreover, details about the test items, test environment, and other references is included here.
Variances: This section of the report highlights any variances or deviations from the test plan or the reference document that might be a concern for the reader or the client. Moreover, resources and documents are offered here to support and explain the deviations.
Comprehensiveness Assessment: The purpose of this part of the format is to evaluate the various testing activities and processes to validate its effectiveness and quality. Additionally, the team evaluates the test coverage and identifies the uncovered attributes.
Summary of Results: Here, a summary of the results is provided. This includes information about the entry and exit criteria as well as the criteria based on which tests are considered passed and failed. Other details offered here includes:
Total incidents.
Resolved and unresolved defects and incidents.
Defects pattern.
Severity & priority of defects.
Evaluation: Based on the earlier assessments and evaluations, the team again evaluates the process of testing and find its limitations, weakness, risks- high & medium, among other things.
Summary of Activities: At the end of the report a summary of all the activities performed by the team and the resources used for the same are reported to offer the team, client, and other stakeholders an understanding of the whole project.
Test Procedure/Script Specification:
Under this document a spectrum of information is provided about how the software will be tested by the tester. From the testing practices, processes, and techniques to the combinations of test cases, the required setup, and the procedure, every minute detail about the process is defined to ensure that the software product is tested efficiently before its release in the market.
Purpose: Offers a record of the various test cases covered by the testing procedure as well as an assessment of the procedure itself.
Special Requirements: Here, the special requirements and specifications regarding software testing are specified by the team, such as stages where test is to be used, type of testing-manual & automated, test environment, etc.
Procedure/Script Steps: The activities associated with the process of software testing are defined here like the log, setup, shutdown, wrap-up, and more.
Test Case Specification:
Test case specification document is another deliverable that is offered to the client with the release of the project. Similar to other documents, test case specification document also follows the format set by the IEEE std 829-1998. It specifies the scenarios that will be tested as well as the purpose of a specific test. Test case specification document is developed during the software testing life cycle by the organization that is responsible for the testing the software product.
Objectives: Defines the main objective of software testing as well as the techniques and methods used to achieve the specifications stated by the client before the inception of the project.
Preconditions: The resources, documents, and materials required for the process testing are stated here. These include various guides, tools, requirement and design specification documents, and more.
Input Specifications: It offers a description of the inputs required for test execution.
Output Specifications: These are the specifications that verify the executed test cases and compare them to expected outputs.
Postconditions: Here, the environment and other procedural requirements are stated by the team, which usually include details about hardware and software, setup, compatibility with various OS and devices, operations verification, etc.
Intercase Dependencies: This is the finals section of the report, wherein the details about any requirements and prerequisites of test cases is defined by the team.
Introduction to 829-2008:
The relevance and significance of the IEEE standard 829-1998 is immense during software development life cycle and it was rigorously followed by engineers all over the world for documentation. However, it has now been superseded by its newer and updated version IEEE standard 829-2008. This new standard helps overcome the few drawbacks of the older version and allows the teams to document the testing process properly.
With the assistance of standard 829-2008 the team could follow a new approach of documentation as well as be more focused on the process of testing, which were usually neglected in the former version. Moreover, another version of this standard (29119) is created by ISO/ IEC, which has superseded all these former standards.
Conclusion:
IEEE standard 829-1998 has played a major role in documenting miscellaneous activities related to software testing, which further helped organizations build a transparent and reliable relationship with their clients and customers. With the assistance of this standard software testers and developers could include important and necessary information in various documents that are prepared after proper evaluation and assessments.
Moreover, by specifying the format for the aforementioned documents it has simplified the way of documenting information and delivering details about the software testing process to the client and other stakeholders of the project.
.png)
.png)
