Testing
Testing consumes 30% to 50% of the effort required to produce a working system.
It's a major project activity that most programmers just don't enjoy. Programmers are not motivated to engage in an activity so redolent with negativity - spending your day looking for nasty ways to break your own code. The core problem is that programmers view their code as inherently beautiful and faultless which makes it difficult for them to find bugs. To make it worse it is generally agreed that testing does not prove the absence of bugs; testing only demonstrates the presence of bugs. Finding thousands of bugs does not mean you have found all the bugs. Absence of bugs does not help either, it could mean that your test cases are ineffective.
So testing's harsh realities, in some respects, parallel those of software development in that both processes exhibit an inherent lack of visibility. For both of them aspects of quality and efficiency are very difficult to observe or predict.
One thing has become clear, if testing is to be effective it has to be performed by someone with a "good attitude". That is they need to not only enjoy testing but also be good at it. Attitudes to testing have been classified and organised into levels of maturity (see sidebar, Attitudes to Testing). Where do we fit in? CA people have always loved to test. Early on we started out with attitudes at maturity level two, for example:
"All programmers are fundamentally damaged human beings who are incapable of producing quality software. The tester's job is to take their miserable attempts at software development, break them into several pieces and hand them back bleeding from all orifices."
This attitude produces results up to a point but over the years, like us, most professional testers evolve a more productive attitude indicative of maturity levels three and four. Today we much prefer to work with developers early in the life cycle to create testable requirements, testable design descriptions and testable code to the point where most of the problems are worked out before the software product hits the test floor. This is by far the cheaper option.
Services
CA develops complete test programs or provides test engineering services to test teams. In these roles we:
- Review requirements, designs and code for testability
- Develop test plans
- Design tests
- Develop test procedures and test cases
- Conduct testing and record results
- Analyse test results and produce test summary reports.
The sidebar: Test Documentation provides a summary of typical test artefacts.
Case Studies
| Boeing
(Strike Jet Missile Program) |
Boeing Australia was contracted to the Royal Australian Air Force to deploy the AGM-142 stand-off missile system on the F/RF-111Cs strike jet. With an 800 lb blast fragmentation warhead or penetrator, the missile was designed to destroy heavily defended targets such as air defence and command-control-communications sites.
CA provided test engineering services to Boeing's system testing group developing system test plans, designs and test
cases. |
| Pacific Link Alliance(Road Tunnel Fire Protection Systems) |
The Tugun Bypass is a 7.5 km motorway providing a high-speed link
between the Gold Coast and northern New South Wales
by bypassing the suburb of Tugun. The bypass includes a tunnel under the
Gold Coast Airport runway. The Queensland Government appointed the
PacificLink Alliance (PLA) to design and construct this motorway at a
cost of $600 million. The PLA comprised Queensland Main Roads,
Abigroup Contractors Pty Ltd and SMEC Australia Pty Ltd.
As a safety measure the road tunnel was fitted with
state of the art intelligent transportation, building management, tunnel
ventilation and fire suppression systems. The tunnel fire suppression systems are managed
with a three level hierarchical control system which, under normal operating conditions, is monitored from the Nerang Traffic Management Centre 29 km from
the tunnel. The construction contract required the fire suppression and smoke extraction systems to comply with the
requirements of IEC 61508. CA developed the system test program for the fire suppression and smoke extraction systems and worked with the test execution team to complete testing. |
| Foxboro(Underground Railway Environmental Control System) |
Hong Kong's Mass Transit Railway (MTR) System underwent a major
refurbishment of its traction power and environmental control systems (ECS).
Foxboro Australia Pty Ltd won a A$70 million contract to provide a
Supervisory Control and Data Acquisition System to control the
distribution of traction power to the trains and to regulate air flow,
temperature and humidity in 37 MTR stations and connecting tunnels.
CA provided testing services to Foxboro's test group developing test designs and test cases and executing unit testing on a major unit of
ECS software.
|
|
Attitudes to Testing
(Increasing levels of maturity)
- Testing is debugging
There is no difference between testing and debugging. Other than support of debugging, testing has no purpose
- Testing proves the software works
The purpose of testing is to show that the software works
- Testing proves the software doesn’t work
The purpose of testing is to make the software fail
- Testing is risk reduction
Testing increases our level of confidence that the software will not fail when delivered to the customer; more testing means less risk
- Testing is a state of mind
Testing is not a single act. It is a mental discipline that results in software that doesn’t need much testing. Testable code has fewer bugs.
Learn more ⇒
Test Documentation
Learn more ⇒
|
