(Alias: Test Plan, Test Design, Test Procedure, Test Case, Test Incident Report, Test Log, Test Summary Report)
If it's not written down it doesn't exist.
- Anon
Test documentation is the complete suite of artifacts that describe test planning, test design, test execution, test results and conclusions drawn from the testing activity. As testing activities typically consume 30% to 50% of project effort, testing represents a project within a project. Testing activities must therefore be fully documented to support resource allocation, monitoring and control. This page identifies the types of documents you need to set up and run your test program and summarises their content.
Tests must be planned and documented to ensure that test coverage is systematic and complete.
Test Plan
The test plan describes the testing process in terms of the features to be tested, pass/fail criteria and testing approach, resource requirements and schedules.
Introduction
Test items
Features to be tested
Testing approach
Item pass/fail criteria
Suspension and resumption
Deliverables
Tasks
Environmental needs
Responsibilities
Staffing and training needs
Costs and schedule
Risks and contingencies.
Test Design Description
The Test Design Description refines the Test Plan's approach, identifying specific features to be tested and defining the test cases and test procedures that will be used.
Features to be tested
Test items covered
Feature or feature combinations to be tested
References to software requirements specifications and software design descriptions
Testing approach
Testing techniques to be used
Method of analysing test results (e.g. automated or manual)
Reasons for selection of various test cases
Environmental needs of test cases
Test identification
For each feature to be tested provide:
Test procedure identifiers and descriptions
Test case identifiers and descriptions
The pass/fail criteria.
Test Procedure Description
The Test Procedure specifies the steps for executing a set of test cases.
Purpose
Test items to be tested
Expected outcomes
Test cases to be exercised
References to test item documentation
Special requirements
Prerequisite procedures
Environmental requirements
Special skills
Procedure steps
Set up. Preparation for the procedure
Start. Starting the procedure
Proceed. Procedure execution
Measurement. Test measurements
Shutdown. Suspending testing
Restart. Restarting the procedure
Stop. Bringing test execution to an orderly halt
Wrap up. Restoring the environment
Contingencies. Dealing with anomalous events
Logs. Logging test results.
Test Case Description
The Test Case Description describes a single instance of a test in terms of data inputs, expected outcome, required test environment and test procedures.
Test items
Input specifications
Output specifications
Environmental needs
Special procedural requirements/rules
Intercase dependencies.
Test Incident Report
The Test Incident Report documents any event that occurs during testing which requires investigation.
Summary
Incident summary
Test item identification
Test procedure and test case identifier
Test log reference
Incident description
Date and time
Expected results
Actual results
Anomalies
Procedure step
Environment
Attempts to repeat
Testers and observers present
Impact
Impact on other test activities.
Test Log
The test log provides a chronological record of details about the execution of test procedures, records success or failure and describes anomalies experienced.
Description
Test item identification
Test environment description
Activity and event entries
Date and time
Author
Test procedure identifier
Staff present
Pass/fail
Error messages generated
Environmental information
Anomalous events
Test incident report identifiers.
Test Summary Report
The Test Summary Report summarises the results of testing and evaluates the test item based on these results.
Summary
For each test item provide:
Test item identifier
Summary test item evaluation
Test environment
References to test documentation
Variances
From design specifications
From test plans, designs, procedures or test cases
Comprehensive assessment
Comprehensiveness of the test
Features that were not sufficiently tested
Summary of results
Tests incidents resolved
Tests incidents not resolved
Evaluation
Limitations
Test performance (pass/fail)
Potential reliability and maintainability
Summary of activities
Resource consumption
Performance against plan.
Case study: Two Angry Men
By Les Chambers
One day I was sitting at my desk minding my own business when I looked up and found I was surrounded by two angry men. A software component I had unit tested on the test floor failed when it was installed on-site. The failure was due to an obvious bug that should have been caught in testing.
Feeling the onset of a mild case of panic I reached for my test records. I had tested that unit about four months previously so my memory was hazy. The angry men were right, it was an obvious bug that should have been caught in testing. How could I have been so stupid?
I laid my hands on the appropriate documentation. What a relief! My records showed that I had run a unit test case for the function in question and it had passed. It looked like the problem was in integration testing. An interaction with another system element that I could not cover with a unit test had caused the failure.
I know, this did not help my assailants who's anger was entirely righteous. Corrective action was going to be expensive in terms of dollars, time and credibility with the customer. The failed software component would have to be fixed in the development shop, re-unit tested, re-integration tested, re-system tested, re-integrated on-site and go through acceptance testing again - while everyone including the customer sat around waiting.
The only positive was for me, at least the anger was focused somewhere else; which just goes to show, quite apart from being the right and professional thing to do, maintaining accurate, bullet-proof test documentation will one day, I am sure, help you avoid unfortunate and unnecessary unpleasantness.
Member Comments
22 Comments
16 member ratings
✭ ✭ ✭ ✭ ✩
RE Definition: Test Documentation
no
By Bhatti » Mon 21-May-2018, 14:45, My rating: ✭ ✩ ✩ ✩ ✩
