The authors describe three ways to measure tool use: (1) controlled experiments, (2) measure programmers to determine which tools they naturally use, and (3) survey their preferences. This paper describes an experiment based upon this third method. From this reviewer’s experience, the first technique is quite expensive and the second technique does not clearly identify contributory factors. However, the third technique depends heavily on the knowledge of those being surveyed.

The authors reduced the 400 tools in the National Bureau of Standards database into 20 classes and gave 25 expert COBOL programmers pairwise choices of tools which they had to rate. The results must be read with two restrictions: (1) Tools--even by the 20 classes--vary greatly (e.g., only one testing tool mentioned was a test coverage analyzer, and there are various versions of these) (2) More importantly, the background of the programmer influences these. Even though definitions are given, that is no substitute for use. So while most programmers have used a screen editor and know its value, few have used testing tools. It is hard to anticipate the value of a tool you have never used (e.g., try to explain the value of a computer to a prehistoric individual who can barely count).

The results of this paper can be used as a first approximation of useful tools for a programming environment. It is necessary to use programmers well versed in these tools to better refine these results.