Analyzing existing source code to identify relationships between classes might be more accurate using the authors’ new method, normalized software distance, which is introduced, derived, and analyzed in this paper. Correctly identifying relationships between sections of code could be an essential starting point in automating tools for improving testing, maintenance, and other improvements in large software systems. Natural language processing (NLP) has been used to discover relationships between sections of code; however, source code is very different from natural language. Their proposed metric, normalized software distance (NSD), is shown to be superior to general NLP methods.

The authors analyzed a number of currently proposed methods, including latent semantic analysis (LSA) [1], normalized Google distance (NGD) [2], pointwise mutual information (PMI) [3], path-based methods [4,5], information-content methods [6,7,8], and a definition-of-words method [9].

Three software systems were used for comparison. All were in Java. One was a student-developed open-source medical application. It was 47.6 KLOC (thousands of lines of code). The second was a subproject of the Apache Ant project. It was 40.9 KLOC. The third was a financial software package contributed by an industrial partner. Each software system had participants with two or more years of experience with the respective software. The above methods were applied to these three software projects and the results compared to the knowledge of the participants. The evaluation measures were recall analysis and mean average precision. The LSA method was the best of the current methods.

The analysis showed a number of issues. The methods based on external sources were not as good as those based on the source code. However, the sparsity and lack of uniqueness in source code were an issue for code-based approaches, including the perfect term dependence (where two terms only appear together). Their approach, NSD, uses a hybrid technique using the class as the level of granularity for the code analysis. The authors provide a theoretical derivation of their approach. The resulting NSD method involves normalizing the maximums of logs of inverses of Bayesian probabilities of one term given the occurrence of the other. The resulting measure, NSD, shows statistically significant improvement over the other methods on these three software projects. NSD also had the lowest time complexity in both pre-processing and relatedness calculations.

The paper includes an extensive set of references and a thorough explanation of the experiments, results, derivation of the NSD measure, NSD results, limitations, and related work. This paper would be a good technical introduction to the area of semantic relatedness in source code.