The state of Continuous Integration & adoption within the industry.
Implementing Continuous Integration [CI] at an enterprise software organization can be just as controversial as it is valuable. The CI engineering paradigm colloquially aspires to increase software release quality through frequent code integrations (merges), smaller batch sizes, and immediate feedback surrounding changes introduced to a software system. Arguably the most controversial segment of CI surrounds development personnel and their religious views of branching patterns. Developers can be fairly cranky when it comes to fundamental alterations in the way they work. Motivating strong-minded individuals who are introverted and shy to communicate more and expose risk earlier is no easy feat.
The concept of CI received numerous accolades and realized significant traction soon after it was introduced as a development best practice via extreme programming [XP]. Since its inception CI has been spearheaded and evangelized by well-known thought-leaders within the technology industry [Martin Fowler, Jez Humble and James Shore]. With as much press as CI has received, software engineering organizations and technology providers have to this day seen mixed results in encouraging engineering teams to adopt continuous integration best practices. The main area of contention surrounds avoiding the urge to create isolated feature branches, or implement promiscuous integration [PI] practices. Since the invention of GIT, GITHUB and similar Distributed Version Control Systems [DVCS], CI branching strategies and best practices are now becoming an afterthought at enterprise software engineering organizations. CI has always aspired to become a foundational engineering best practice regardless of technology and platform constraints. Teams that practice CI properly have been shown to catch defects earlier in the software development lifecycle [SDLC], and release software with higher quality. So why then is there such contention within the industry?
While CI has increased in notoriety, SaaS providers and open-source software developers have introduced countless DevOps related automation solutions. So much so that there is even a periodic table, which details the numerous solutions [shown below, provided by Xebia labs] on the market today.
While there exists a multitude of CI software solutions, very few of them document CI best practices (branching, merging, code conflicts etc.) and show the value that these practices provide to engineering and businesses. Software providers are also advertising the various branching methodologies that their CI solutions support. Unfortunately this creates a rift between what was a once well-defined practice and what is now implemented at various software shops around the world. Some engineering professionals will even question your sanity when a proposal around mainline development surfaces. It is important to note that the tools these organizations have created do add ancillary technology support for Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment practices; however they do not represent the fundamental CI engineering practice.
Web accessible documentation for CI software solutions includes a bare minimum amount of industry research on CI. Their advertising of bullet point features continues to drive their wares to the top of any web based search results. This is summarily due to a large marketing budget, which can almost always generate more backlinks and SEO gander than a blogger or book can. This problem seems to be compounded by the fact that less and less people read books and more and more people rely on quick Google results. The end result of this misappropriated research is confusion in the definitions surrounding CI, CD best practices.
The originating founder of Continuous Delivery aptly attempted to address some of this confusion in his 2007 Spark conference speech on Continuous Delivery. Jez stated:
“Continuous Integration is hard. It’s not running your CI tool on your feature branches. It’s not running your CI tool and having the build be red all the time; but no one cares. ” – Jez Humble
While his efforts and those of other CI ‘purists’ have not gone completely unnoticed, it’s now becoming increasingly difficult to differentiate tooling documentation for CI and documented research surrounding continuous practices as a engineering discipline. Let’s look at Google for an example. Via Google (aside from the Wikipedia article and a few blog posts) the first page of search results for “Continuous Integration” provides links to at least 5 CI product offerings. Within these results, each company selling a CI product provides a slightly different definition and approach to ‘CI’, which is tailored to fit their specific product. None of them seem to address the ‘integration’ practice required as part of continuous integration.
The ongoing confusion and mixed documentation leads many technology professionals to erroneously believe that CI is defined as the adoption of a tool OR simply represents auto-builds on feature branches. Modern DVCS technologies such as GIT have seemingly compounded this issue by providing easy feature branching mechanisms and have integrated code review practices into feature branches. This mendacious concept continues to spread throughout the ranks of engineering and software technology professionals, and is a reflection of the improper research conducted by marketing professionals. In essence the practical implementation requirements of Continuous Integration or Continuous Delivery seem get lost amidst the ocean of commoditized CI tools.
To help clarify the ambiguity surrounding CI and CD let’s take a quick look at the definitions of each.
Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies to a shared mainline several times a day. – Source: Wikipedia + Martin Fowler
Continuous Delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time. It aims at building, testing, and releasing software faster and more frequently.
Now that we have CI/CD and their respective definitions, let’s clarify the proposition a bit further. At this point I am going to draw a line in the sand. An organization cannot purport to be practicing CI with feature branches in play. This is because the ‘integration’ practice is missing and barriers have been erected that prevent the continuous flow of changes. Integration in the CI realm is synonymous with merging code and integrating work with the larger engineering team. In addition an organization cannot leverage continuous delivery unless continuous integration has been achieved first. This is by definition. Continuous Delivery is an extension of Continuous Integration and represents the furthering of CI practices to expand beyond simply merging and building code frequently. Into the realm of proving that the software is releasable through automation. The idea here is that many micro sized changes to a software system have greater value and less risk than obtuse and large commits that were created in isolation and outside of the realm of normal visibility.
Many engineers and technology professionals will argue that using many short-lived feature branches managed by a well-disciplined team represents the practice of CI. While this practice is better than creating long-lived feature branches it is more commonly known as promiscuous integration [PI]. The reasoning is that the practice needs to take into account the following points.
- The amount of time a feature branch will live
- The frequency in which the feature branch is merged into the mainline
- Possible syntactic or functional code collisions that are lurking in other feature branches being developed in parallel
- The delay in merge due to a longer than expected code review process
- The high frequency of communication and collaboration, which is required by CI
- The amount of drift between the feature branch and other parallel feature branches being worked on in comparison to the mainline (releasable entity).
- The encouragement and discipline required by mainline development to branch in code (using feature toggles or branching by abstraction) instead of in VCS.
The encouragement and discipline required by mainline development is probably the hardest aspect of CI. To successfully adopt and implement continuous integration and continuous delivery in an enterprise organization will require trust, the removal of process barriers and the creation of a blameless culture. These concepts are becoming more and more widespread as software engineering matures, complexity is traded for simplicity and releases are decoupled from deployment.