Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
Comparison of adaptive random testing and random testing under various testing and debugging scenarios
Liu H., Kuo F., Chen T. Software--Practice & Experience42 (8):1055-1074,2012.Type:Article
Date Reviewed: Mar 1 2013

Testing and debugging remain the most practical means of assuring the quality of software. In particular, partition testing and random testing are widely adopted in test case selection. According to classic studies, partition testing is better than random testing by only a small margin in many cases, but requires considerably more overhead. Random testing is, therefore, popular as a simple alternative.

In previous work [1,2], Chen et al. introduced the innovative idea of adaptive random testing (ART) to further enhance random testing in test case selection. The fundamental concept is to improve the diversity or even spreading of test cases in the input domain. ART improves over random testing by more than 40 percent in terms of the F-measure, which is the number of test cases expected to reveal the first failure.

In real-life projects, however, the software being tested may exhibit numerous failures. It is more customary to continue testing with a view to identifying more failures before debugging commences. This paper proposes a new metric to gauge the number of test cases expected to reveal the first n failures. Using this new Fn-measure, the authors study the empirical results of two standard scenarios, in which testing and debugging are performed either simultaneously or in turn. They find ART to be statistically more effective than random testing in relation to the Fn-measure when testing and debugging are conducted in turn.

For readers interested in comparing test case selection approaches, this paper provides innovative insights and practical solutions. The Fn-measure will undoubtedly become a new yardstick for test case selection. Further studies of its application to test case prioritization and fault localization should provide additional insights.

Reviewer:  T.H. Tse Review #: CR140977 (1306-0517)
1) Chen, T. Y.; Leung, H.; Mak, I. K. Adaptive random testing. In Proceedings of the 9th Asian Computing Science Conference (ASIAN 2004) Maher, M. J., Eds. Springer, 2004, 320–329.
2) Chen, T. Y.; Kuo, F.-C.; Merkel, R. G.; Tse, T. H. Adaptive random testing: the ART of test case diversity. Journal of Systems and Software 83, 1(2010), 60–66.
Bookmark and Share
 
Testing And Debugging (D.2.5 )
 
Would you recommend this review?
yes
no
Other reviews under "Testing And Debugging": Date
Software defect removal
Dunn R., McGraw-Hill, Inc., New York, NY, 1984. Type: Book (9789780070183131)
Mar 1 1985
On the optimum checkpoint selection problem
Toueg S., Babaoglu O. SIAM Journal on Computing 13(3): 630-649, 1984. Type: Article
Mar 1 1985
Software testing management
Royer T., Prentice-Hall, Inc., Upper Saddle River, NJ, 1993. Type: Book (9780135329870)
Mar 1 1994
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy