The C preprocessor is often used to configure a program for different environments. A combination of flags denotes a particular environment, controlling conditionals and macros to generate an appropriate version of the program. Generally, only a few of the many possible versions are actually used in practice, and those are the ones that developers generate and test. Untested combinations of flags may result in syntactically incorrect programs. The authors of this paper use a parser that checks all possible versions of the source code to evaluate the extent of preprocessor-based syntax errors in real systems.

The experiment was extensive, using 41 systems ranging in size from 2,000 to 1.5 million lines of code as experimental subjects. These subject programs had been placed under version control by their authors, so the experimenters were able to look at the commit in which a preprocessor-based syntax error was introduced and the commit in which it was corrected. The paper poses and answers seven interesting questions, such as, “How do developers introduce preprocessor-based syntax errors?” and “What types of preprocessor-based syntax errors do we find in practice?”

I found both the methodology and the results interesting. The authors’ use of version control history to explore the genesis and persistence of these errors is compelling, and the low number of actual errors found indicates that we do a pretty good job of avoiding them. The paper is easy to read, and there are appropriate references to the tools used and to the topic in general.