Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization
Xie X., Chen T., Kuo F., Xu B. ACM Transactions on Software Engineering and Methodology22 (4):1-40,2013.Type:Article
Date Reviewed: Dec 13 2013

Spectrum-based fault localization is a popular technique in automatic program debugging. Researchers analyze the distribution of pass and fail cases in program testing using different risk evaluation formulas, and validate how their proposals are better than earlier work via empirical studies. In this paper, the authors propose a theoretical framework to compare 30 risk evaluation formulas in terms of the percentage of code examined before a fault is identified. They rank the formulas using “better” and “equivalent” relations. Only five formulas are proven to be the most efficient. Many of the best-known formulas are not among them.

There is an unhealthy tendency toward empirical studies in software testing and debugging research. Researchers use hypothesis testing to determine whether their proposal is better than that of their predecessors. Reviewers demand more subject programs and larger test pools for further validation. It is refreshing to see that the authors of this paper do not simply rely on empirical studies, but prove mathematically whether various proposals have hit their mark.

This paper is not the only example of the successful application of mathematical theory by Chen’s research group. Chen and Merkel prove in one paper [1] that no test case generation technique can be better than random testing by more than 50 percent. Hence, their proposed adaptive random testing technique is close to this theoretic limit. In another paper [2], Chen and Yu prove that their proposed proportional sampling strategy is the only partition testing strategy that ensures that the probability of finding at least one failure is no lower than random testing for any program. Understandably, some researchers are disgruntled because these theoretical results stop them from making further incremental proposals.

Reviewer:  T.H. Tse Review #: CR141805 (1402-0139)
1) Chen, T. Y.; Merkel, R. An upper bound on software testing effectiveness. ACM Transactions on Software Engineering and Methodology 17, 3(2008), 16:1–16:27.
2) Chen, T. Y.; Yu, Y. T. The universal safeness of test allocation strategies for partition testing. Information Sciences 129, 1-4(2000), 105–118.
Bookmark and Share
  Featured Reviewer  
 
Testing And Debugging (D.2.5 )
 
Would you recommend this review?
yes
no
Other reviews under "Testing And Debugging": Date
Software defect removal
Dunn R., McGraw-Hill, Inc., New York, NY, 1984. Type: Book (9789780070183131)
Mar 1 1985
On the optimum checkpoint selection problem
Toueg S., Babaoglu O. SIAM Journal on Computing 13(3): 630-649, 1984. Type: Article
Mar 1 1985
Software testing management
Royer T., Prentice-Hall, Inc., Upper Saddle River, NJ, 1993. Type: Book (9780135329870)
Mar 1 1994
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy