Objective Evidence

(Alias: physical proof)

One test result is worth 1000 expert opinions.
- Wernher von Braun, Rocket Scientist

Objective evidence is any documented statement of fact, other information or record, either quantitative or qualitative, pertaining to the quality of an item or activity, based on observations, measurements or tests which can be verified.

Examples of Objective Evidence

Test log
Test report
Review report
Non-conformance report
Witness statement
In information systems: audit trail
Quality metric (example: in software development, code defect density - defects per thousand lines of code)

Using Objective Evidence in IT Audits

The objective of an audit is described by ISO 9001¹ clause 8.2.2 Internal audit:

"The organization shall conduct internal audits at planned intervals to determine whether the quality management system
a) conforms to the planned arrangements (see 7.1), to the requirements of this International Standard and to the quality management system requirements established by the organization, and
b) is effectively implemented and maintained."

Making Audit Findings Credible

Any anomalies identified by an audit are documented in non-conformance reports. For a non-conformance report to be credible it must be backed up by objective evidence (refer example in glossary item: Non-conformance Report). Anomalies that lack credibility are unlikely to be followed up by management thus defeating the main purpose of an audit, that is: to identify problems and trigger corrective action on the quality management system.

Getting to the Truth

When conducting audit interviews, wherever possible, professional auditors seek out objective evidence in support of the assertions of auditees.

Example:
Auditor: Did you test the system?
Auditee: Yes.
Auditor: Sounds good, can you please provide me with your test plans, test designs, test cases, test results, test reports together with records of the resolution of all test anomalies identified.

A Non-example of Objective Evidence

Case study:
An IT auditor reviews the design process in a software development shop. In his audit report he concludes that: "... the design process is inadequate and produces poor quality design descriptions."
In the absence of objective evidence to back up this statement these assertions are open to dispute as they are based on one individual's personal opinion. In this case it is unlikely that any corrective action would be taken by management because the auditor's pronouncement lacks specificity and proof.

A more professional approach would have the auditor providing objective evidence in support of his conclusions. For example he could provide measurements of the degree to which:

The design descriptions complied with company standards (standards compliance)
The design descriptions satisfied requirements documented in software requirements specifications (requirements traceability)
The design was implemented in the code (code traceability).