Given the vitality of replication experiments in other fields of empirical research, it is surprising that more attention has not been paid to replication of studies in software engineering. The authors of this paper cite a number of reasons for that, including lack of guidelines and incentives, and seek to assist in moving forward with replications in this field by performing a systematic mapping study of such replications over almost two decades of published software engineering research. The results of this study are somewhat surprising: only 133 replications were performed between 1994 and 2010, with 70 percent of those published after 2004, largely focused on a relatively small subset of the Software Engineering Body of Knowledge (SWEBOK).

The authors manage to present a substantial study in a concise and organized format, making it a useful reference for future research. The results of the study are presented systematically in answer to seven specific questions in section 4 regarding the scope and frequency of replications. The discussion and conclusions in section 5 and 6, respectively, are worth careful consideration.

This paper will be of primary interest to those involved or interested in research in empirical software engineering.