Hardly ever is a major commercial computer program released to the public without significant bugs. Many software producers have turned to “public betas” as a way of enlisting the user community in the testing process, but the high bug rate persists. The most reliable programs we have, such as the software that controls telephone exchanges, are developed under very different circumstances than are the operating systems and applications, sold on the open market, that account for most of the software now being produced.

Murphy attempts to address the question of how to develop relatively bug-free commercial software. He pursues two parallel themes: providing a set of software testing tools, and offering advice on the managerial issues relating to the testing process. He implicitly assumes that the program being tested is a Windows program written in C or C++--certainly a common and practical case--although much of the content of the book could translate to other environments.

The C source code for the six tools is included on a diskette that accompanies the book.

AssertLib is a collection of C macros for checking preconditions, postconditions, and invariants of function calls. The normal C assert() macro dumps a message to standard output and terminates the program; the Assert() macro offers options such as continuing execution or disabling the check altogether (for production code).

ResTrack is a collection of macros and supporting routines for checking resource allocation and freeing, particularly for heap storage. The macros redefine the usual malloc(), free(), and strdup() functions to alternate versions that check for writes beyond buffer boundaries, memory leaks, and improper freeing.

UTCGen is a test case generator that extracts function prototypes and suggests test cases based on the data types of the functions’ parameters. For instance, for each string parameter it suggests trying a null string, a one-character string, and a multiple-character string.

UTCLang is an interpreted object-oriented scripting language with a Smalltalk-like syntax for writing scripts to be used for testing subsystems. The built-in classes of  UTCLang  are normally augmented by classes that correspond to the C++ classes of the subsystem being tested.

UTCCover is a collection of macros for checking branch, function call, and loop coverage. It does not provide statement coverage or multiple condition coverage.

WinCapT is a tool for capturing sessions under Windows so that they can be played back and examined. The tool uses the Windows journaling function. It provides essentially the same functionality as the Windows Recorder. In addition, it enables the tester to capture the control states (checkbox settings, field text, and so on) of the window under the cursor.

These tools might sometimes be useful, but they do not seem very sophisticated. For instance, they enable you to test whether a call to open a file returns zero, but not whether the file was actually opened. The test cases suggested by UTCGen do not consider the semantics of the parameters being tested. The UTCCover tool provides no help with the many bugs that occur only when code fragments are executed in a particular sequence; testing tools that employ dataflow analysis to partition the set of all possible paths can often expose such bugs. The tools that use the C preprocessor may run into difficulties with programs that themselves use the preprocessor extensively. Also, Murphy provides nothing to help organize the output of a test suite so that erroneous results can be spotted instantly.

Discussions of methodology are scattered throughout the book. The description of each tool is accompanied by a discussion of methodological issues related to that tool. In addition, there are chapters on “Quality Control During Development” (including code reviews and coding standards), “No Code Is an Island” (on unit testing versus system testing, and iterative development), “The Final Stand” (on how to organize system testing, who should do it, and maximizing benefit from one’s efforts), and “Putting It All Together” (quality after delivery, bug tracking, and continuous improvement).

Unfortunately, the book uses specialized terminology (most of it apparently from the Windows development environment) without defining it. The glossary is short and wholly inadequate. Furthermore, the choice of what terms to include seems quirky. For instance, the glossary defines “bug rate,” “McCabe’s cyclomatic complexity,” and “invariant,” but fails to define “unit testing,” “makefile,” or “framework,” the last of which is a critical concept at several points in the exposition.

The natural question to ask is whether using Murphy’s tools and philosophy is likely to make much difference in a commercial software project. I am skeptical, particularly since he does not mention any experience in using these tools other than to test the tools themselves. AssertLib and ResTrack seem more useful than the others, because they address specialized but important tasks. Although the material in this book and on the accompanying diskette has some value, it is far from being the silver bullet that every software developer would love.