To what extent are good practices followed when writing JUnit tests? To answer this question, an automated static analysis of 112 Java repositories from the Qualitas corpus was conducted. Of the 112 projects, 70 were found to have used JUnit as their testing framework. Over 55000 JUnit tests were analyzed.

Some 61 percent of testing methods were found to contain more than a single assertion, violating the single checkpoint principle. Almost 31 percent of tests were found to be nonlinear, being made up of loops or if statements. Some 77 percent of assertTrue and assertFalse assertions were found not to have any customized error message. Little use was made of Hamcrest, a third-party assertion library.

The investigator correctly draws attention to the main validity threats. First, the Qualitas corpus dates from 2013. Many changes could have occurred since then, such as the adoption and use of third-party assertion libraries. Second, many systems were found to be using their own assertion methods, which were not part of the analysis. A number of customized assertion methods should have been manually analyzed to provide the reader with insights into their importance. Also, some indication should have been given to the alternative testing frameworks (such as TestNG) used in the other 42 projects.

The conclusion drawn is sound: good practices were often not followed. This paper is recommended to those who work in software quality assurance.