Computing Reviews
Today's Issue Hot Topics Search Browse Recommended My Account Log In
Best of 2016 Recommended by Editor Recommended by Reviewer Recommended by Reader
Sentiment analysis for software engineering: How far can we go?
Lin B., Zampetti F., Bavota G., Di Penta M., Lanza M., Oliveto R.  ICSE 2018 (Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden,  May 27-Jun 3, 2018) 94-104. 2018. Type: Proceedings
Date Reviewed: Aug 15 2019

Sentiment analysis continues to be successfully applied to consumer reviews. In other areas, the challenges involved can prove insurmountable. A failed attempt to successfully apply sentiment analysis is reported in this negative results paper.

The investigators designed a system that would have been capable of recommending software libraries based on text extracted from Stack Overflow discussions. They adopted a “state-of-the-art approach based on a recursive neural network,” Stanford CoreNLP, to analyze the extracted text for sentiments. A training set involving the manual labeling of sentiments required some 90 hours of work to build. Despite this best practice effort, as the final row of table 2 indicates, overall “precision and recall in detecting positive and negative sentiments [was less than] 40 percent.”

As a follow up, the investigators evaluated five sentiment analysis tools on three datasets (Stack Overflow discussions, app reviews, and JIRA issues). Results for the Stack Overflow discussions dataset were, as before, not acceptable. For the app reviews dataset, however, results were acceptable. This success was attributed to the fact that the reviews were similar to consumer reviews in which opinions are clearly expressed. Results for the JIRA issues dataset were also acceptable, but the interpretation is complicated by the fact that there are no neutral sentences in this particular dataset. The investigators come to the inevitable conclusion that opinion mining a dataset comprising developer discussions of technicalities is obviously a very difficult challenge.

This paper provides several useful insights and is strongly recommended to those working on sentiment analysis.

Reviewer:  Andy Brooks Review #: CR146654 (1911-0393)
Bookmark and Share
  Reviewer Selected
Editor Recommended
Featured Reviewer
General (D.2.0 )
Natural Language Processing (I.2.7 )
Would you recommend this review?
Other reviews under "General": Date
Handbook of software engineering
Cha S., Taylor R., Kang K.,  Springer International Publishing, New York, NY, 2019. 524 pp. Type: Book (978-3-030002-61-9)
Feb 14 2020
Simplicity is complex: foundations of cyber-physical system design
Kopetz H.,  Springer International Publishing, New York, NY, 2019. 148 pp. Type: Book (978-3-030204-10-5)
Jan 23 2020
 On the impact of programming languages on code quality: a reproduction study
Berger E., Hollenbeck C., Maj P., Vitek O., Vitek J.  ACM Transactions on Programming Languages and Systems 41(4): 1-24, 2019. Type: Article
Nov 21 2019

E-Mail This Printer-Friendly
Send Your Comments
Contact Us
Reproduction in whole or in part without permission is prohibited.   Copyright © 2000-2020 ThinkLoud, Inc.
Terms of Use
| Privacy Policy