Can knowledge contained in natural language program comments be automatically exploited to detect bugs in source code? Tan et al. seek to answer this question by first constructing a program-comment analyzer, capable of detecting rules concerned with locks for protecting access to shared variables. The analyzer exploits existing natural language processing (NLP) techniques and makes use of several heuristics. Then, the authors construct a checker: a static analysis tool that takes as input the detected lock-related rules and program source code, and outputs reports of inconsistencies, such as a function being called without acquiring a lock, when the associated function comment clearly states that a lock should be acquired.

A preliminary study of Linux source code and comments results in the automatic detection of 538 lock-related rules and 12 bugs, two of which are reported as confirmed bugs.

As a short proof of concept paper, several details and analyses are frustratingly missing. All that is said about the solution to the synonym problem is the use of keyword variants; the accuracy of the analyzer and checker are left unestimated. Can existing static analysis tools for source code detect some, or all, of the 12 bugs found through program comment analysis?

Despite the paper’s shortcomings, this line of research could be extremely profitable, and lead to the full exploitation of knowledge contained in natural language program comments. As such, I strongly recommend this paper to the software maintenance community.