Markus Borg and Leif Jonsson, Tiny Transactions on Computer Science, Volume 3, 2015

Body

Humans obscured by bug overload, but machine learning benefits from plentiful training data. Practitioners confirm value of developed tools.

Abstract

Issue management, a central part of software maintenance, requires much effort for complex software systems. The continuous inflow of issue reports makes it hard for developers to stay on top of the situation, and the threatening information overload makes activities such as duplicate management, Issue Assignment (IA), and Change Impact Analysis (CIA) tedious and error-prone. Still, most practitioners work with tools that act as little more than issue containers. Machine Learning encompasses approaches that identify patterns or make predictions based on empirical data. While humans have limited ability to work with big data, ML instead tends to improve the more training data that is available. Consequently, we argue that the challenge of information overload in issue management appears to be particularly suitable for ML-based tool support. While others have initially explored the area, we develop two ML-based tools, and evaluate them in proprietary software engineering contexts. We replicated [1] for five projects in two companies, and our automated IA obtains an accuracy matching the current manual processes. Thus, as our solution delivers instantaneous IA, an organization can potentially save considerable analysis effort. Moreover, for the most comprehensive of the five projects, we implemented automated CIA in the tool ImpRec [3]. We evaluated the tool in a longitudinal in situ study, i.e., deployment in two development teams in industry. Based on log analysis and complementary interviews using the QUPER model [2] for utility assessment, we conclude that ImpRec offered helpful support in the CIA task.