How can one measure the degree of comprehension of computer program code? Researchers studying both novice and expert programmers have been constructing their own answers to this question. The authors of this paper propose an answer to this question that they hope will have far-ranging applications. They describe a scheme for coding program summary statements, in an attempt to get “more insight into the types of information that programmers glean from a program” (page 171).

Research participants are asked to write summaries of computer programs, after studying them. The summaries are analyzed based on the information types provided in the summary statements, and based on how objects are described there. The information types classification scheme describes 11 categories of statements, ranging from large-scale events, for example, describing the purpose of the program, to small-scale events, for example, incrementing a counter. The object classification scheme describes seven categories, encompassing information about the extent to which the object is a real-world object, or an object only in the program. Some aspects of the authors’ coding scheme are based on Pennington’s methods [1]. The authors discuss the new scheme’s use in a study where novice programmers were asked to summarize programs written in a visual programming language. The authors end with a request for comments, and suggestions regarding the coding scheme.

The coding scheme is carefully described and credible. Its real test will come when it is applied by people other than its originator. As with any coding scheme, the extent to which it serves its purpose will be heavily determined by its inter-rater reliability. The authors tell us this is currently being measured. I look forward to the time when the use of some variant of this scheme is commonplace, and when researchers can concentrate on studying program comprehension, instead of starting at their own point zero in defining and creating their own tools.