There are two major approaches to measuring how well structured object-oriented computer code is. A question immediately arises: Are these two different families of metrics targeting the same intrinsic properties? One family quantifies the degree of coupling statically from the source code; the other quantifies it dynamically, sometimes using the code’s own test suite as the execution environment. This paper aims to answer the question by testing the effect of runtime coverage on metrics correlation.

The platform for data collection comprises code from several Java benchmarks. Static coupling is measured from disassembled bytecode, and coverage comes from instrumented Java executables. The Java platform debug architecture with ad hoc enhancements provides runtime metrics. The analysis uses statistical tools: principal components, multiple regression, analysis of variance, and hypothesis tests. The results show that some dynamic metrics are more than just surrogates for static coupling--they capture different information. Runtime coverage enhances the prediction of some, but not all, dynamic metrics.

This work lays another brick in the incomplete edifice of code analysis. The experimental techniques appear to be robust, and the analysis is carefully designed. Yet, statistics tools often show only what the designer is looking for. The failure of some dynamic metrics to show meaningful behavior is dismissed by the authors as “expected.” This is perhaps misleading: erratic behavior may show limitations of the study. The metrics may insist on properties not properly defined, captured, or made visible here. Altogether, this work is a step in the right direction, with some speculative reservations.