The result of a single experiment is rarely believed. Mistakes might have been made when designing the experiment, conducting the experiment, or analyzing the data. A result becomes an accepted fact only after others have successfully redone the experiment. To clarify the nature of the redoing of an experiment, five terms are proposed and discussed.

Repetition is proposed as meaning to redo the experiment exactly using the same artifacts. Replication is proposed as meaning to redo the experiment but having access only to the descriptions of the artifacts. Variation is proposed as meaning to redo the experiment with controlled modifications to establish the scope of the result. Reproduction is proposed as meaning to redo the experiment with conceptually similar artifacts. Corroboration is proposed as meaning to provide evidence in support of the result of the experiment by using a different approach. Section 9 contains an example about a caching experiment that succeeds in illustrating the use of these five terms.

There are many useful discussions. For example, the conditions for exact repeatability are enumerated along with the impediments to achieving exact repeatability. Because of the transient nature of independent repositories of experimental software and data, a suggestion is made that such repositories are best curated by professional organizations. The discussion on meta-analysis, while useful, should have been expanded upon by drawing on the lessons learned from medical research.

The case is made for use of the five terms. This paper is recommended to all those engaged in experimental work.