Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Review Help
Search
Performance monitoring in SAP HANA’s continuous integration process
Rehmann K., Seo C., Hwang D., Truong B., Boehm A., Lee D. ACM SIGMETRICS Performance Evaluation Review43 (4):43-52,2016.Type:Article
Date Reviewed: Jun 23 2017

Setting up a performance test and validation environment for every source code change is known to be prohibitively expensive. Conversely, SAP HANA’s continuous integration (CI) case suggests that such tests need to be done for all changes being pushed into the mainline code base.

This study divided the CI testing pipeline into two main categories: performance and functional. The latter is usually defined as checking software to make sure that it has all of the required functionality. These tests confirm that the system does what users are expecting it to do. Performance tests aim to verify the speed or effectiveness of a software program. Hence, it poses major challenges to be continuously done, such as the amount of time and computational resources required, because the pace of change is too high to be integrated and tested.

To allow performance regression testing, the authors introduce the concept of performance barriers. Such barriers are performance tests that execute against topic branches of the source code instead of the release branch. With such an approach, they were able to keep developers more efficient because the release branch might be hardly broken.

Although it is appealing to have continuous performance testing, such an approach might put a burden on the merging process and on test configuration. Merging many topic branches and dealing with a multitude of configuration files sounds too error prone. Not accelerating performance activities in software development could cause even worse problems; ultimately, “speed drives quality, rather than being detrimental to it” [1].

Reviewer:  Klerisson Paixao Review #: CR145369 (1709-0615)
1) Bosch, J. Speed, data and ecosystems: excelling in a software-driven world. Keynote given at the Evaluation and Assessment in Software Engineering Conference (EASE), Karlskrona, Sweden, 2017. http://ease2017.bth.se/speakers/keynotes.
Bookmark and Share
  Reviewer Selected
 
 
Metrics (D.2.8 )
 
Would you recommend this review?
yes
no
Other reviews under "Metrics": Date
A comparison of time domains for software reliability models
Musa J., Okumoto K. Journal of Systems and Software 4(4): 277-287, 1984. Type: Article
May 1 1985
On software equations
Král J. Information Processing Letters 19(4): 191-196, 1984. Type: Article
Jun 1 1985
Software metrics: establishing a company-wide program
Grady R., Caswell D., Prentice-Hall, Inc., Upper Saddle River, NJ, 1987. Type: Book (9789780138218447)
Apr 1 1988
more...

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright 1999-2024 ThinkLoud®
Terms of Use
| Privacy Policy