Email this page to a friend   

Email to a friend

IT Quality Control

(Contrast with: quality management, quality assurance)

If we deliver on time, but the product has defects, we have not delivered on time.
             - Philip Crosby, quality control manager of the Pershing missile program

IT Quality control is the process of testing software intensive systems to uncover defects and hence measuring actual quality. In the software development context test candidates can be specifications, design descriptions, code listings, executable software modules, units, subsystems or complete systems.

Quality Control Activities

Quality control activities should occur throughout the software development life cycle. The earlier defects are discovered the less expensive they are to fix.

Quality control activities include:
  • Defining and classifying the severity of defects
  • Inspecting documents
  • Inspecting code (either manually or via automatic static code analysis)
  • Testing executable software. For example: module, unit, integration, system and acceptance testing
  • Recording of defects
  • Setting quality control limits outside of which corrective action must be taken (refer figure: Run Chart, upper and lower control limits)
  • Tracking corrective action on defects
  • Defect data analysis - e.g. tracking defect trends over time (refer figure: Run Chart).

Note that quality control metrics are used as an indicator of software development process effectiveness and act as a trigger for process improvement.

Measures of Software Quality

The most common measure of software quality is defect density expressed as defects per thousand lines of code. Defect density is measured at various points in the software product's life cycle. ISO/IEC 9126 Software engineering - Product quality1 provides many additional measures.

The defect density of delivered software is a simple number by which a software development organisation can be judged. All software development managers should know how their group rates.

Metric Formula Significance
Code defect density
(in system test)
NDT/KLOC Indicates the quality of the code presented to the system test group
Code defect density
(in operation)
NDO/KLOC Indicates the quality of the code delivered to the customer
Defect removal efficiency NDT/(NDT + NDO) Indicates the effectiveness of the test function in removing defects from the delivered software product

KLOC = Thousand Lines of Code
NDT = Number of Defects Detected in Testing
NDO = Number of Defects Detected in Operation

Software Quality Benchmarks

When defining software quality control limits, what are reasonable values for defect densities? When should we decide that our software development process is out of control and needs serious improvement? At what point should a customer reject a custom developed software product if failures are experienced in operation?

This is a difficult question as tolerable defect densities in software vary widely as a function of application type and criticality. For example we routinely tolerate high defect densities in word processing software whereas the avionics software that controls flight surfaces in an aircraft must have extremely low (or zero) defect densities in delivered software. Valid comparisons of defect densities across application domains and organisations are also hampered by variations in the process of defect classification and data analysis methods. Organisations are also understandably reticent to air their dirty linen and share defect density metrics.

In his book Code Complete2 Steve McConnell goes out on a limb and provides the following guidelines:

Context Defect Density (defects/KLOC)
In-house Test Delivered Software
Industry average   15 - 50
Microsoft Applications Division 10 - 20 0.5
Clean room development (Harlan Mills3) 3 0.1
Space shuttle software   0 defects in 500,000 lines of code

- Rate this definition.
- Did it help?
- Suggest improvements.
- Request more information.
- Exchange ideas with our member community.

Email to a friend