Zhao and Elbaum seek to quantify a number of the software quality assurance activities used in the development of open source (OS) software, in an attempt to either support or debunk some of the claims made by the proponents of open source development. The data on which the paper’s results are based was obtained by means of a survey (included in the paper), sent to 474 open source project developers; 232 responses were obtained.

The paper includes a myriad of details, which interested readers will find enlightening. One of the highlights is the fact that 77 percent of respondents performed their open source development in their own time, which reinforces the “giving spirit” often put forward by OS promoters.

All the projects surveyed were hosted by either Sourceforge or Freshmeat, both of which provide source code control and bug tracking facilities. These were used by 75 percent and 61 percent, respectively, of all the projects surveyed, with an even higher take up on larger (greater than 10,000 lines of code) projects. This should be compared with the five to 25 percent usage found with more traditional software development models.

On the testing side, the results were less encouraging; less than five percent of the projects used test coverage tools, and 30 percent estimated test coverage at less than 30 percent. In addition, only about half of the projects used regression testing.

There was evidence of valuable user contributions; 20 percent of source changes in almost half of the projects were the result of user feedback, and users discovered 20 to 40 percent of the faults in 20 percent of the projects.

In summary, the authors have identified a number of areas where traditional software quality assurance (QA) activities can learn from open source methods, and vice versa. The paper, which is reasonably well structured and quite easy to read, will be of interest both to software developers and to managers who either already use open source methods, or who are thinking of taking up the baton.