Usage, costs, and benefits of continuous integration in open-source projects

Michael Hilton, Timothy Tunnell, Kai Huang, Darko Marinov, Danny Dig

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continuous integration (CI) systems automate the compilation, building, and testing of software. Despite CI rising as a big success story in automated software engineering, it has received almost no attention from the research community. For example, how widely is CI used in practice, and what are some costs and benefits associated with CI? Without answering such questions, developers, tool builders, and researchers make decisions based on folklore instead of data. In this paper, we use three complementary methods to study the usage of CI in open-source projects. To understand which CI systems developers use, we analyzed 34,544 opensource projects from GitHub. To understand how developers use CI, we analyzed 1,529,291 builds from the most commonly used CI system. To understand why projects use or do not use CI, we surveyed 442 developers. With this data, we answered several key questions related to the usage, costs, and benefits of CI. Among our results, we show evidence that supports the claim that CI helps projects release more often, that CI is widely adopted by the most popular projects, as well as finding that the overall percentage of projects using CI continues to grow, making it important and timely to focus more research on CI.

Original languageEnglish (US)
Title of host publicationASE 2016 - Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering
EditorsSarfraz Khurshid, David Lo, Sven Apel
PublisherAssociation for Computing Machinery, Inc
Pages426-437
Number of pages12
ISBN (Electronic)9781450338455
DOIs
StatePublished - Aug 25 2016
Event31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016 - Singapore, Singapore
Duration: Sep 3 2016Sep 7 2016

Publication series

NameASE 2016 - Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering

Other

Other31st IEEE/ACM International Conference on Automated Software Engineering, ASE 2016
CountrySingapore
CitySingapore
Period9/3/169/7/16

Keywords

  • Continuous integration
  • Mining software repositories

ASJC Scopus subject areas

  • Software
  • Computational Theory and Mathematics
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'Usage, costs, and benefits of continuous integration in open-source projects'. Together they form a unique fingerprint.

Cite this