Managing Your Build Pipeline More Effectively

Several years ago, I joined a team in their early days of developing a claims product. When I came onboard, the team was small but was expected to grow significantly. While the analysts and business leads were busy identifying the product scope, the technical members of our team focused on the platform. My role was to ensure a productive environment for the developers, and my biggest challenge was figuring out how a team of 30 developers could all work on the same code base.

As my prior experience, had consisted only of internal research projects with one or two developers, I needed to find an effective solution that would meet our current and future needs. Luckily for me, the project architect sent me ‘Continuous Integration’, the now-landmark article by Martin Fowler (the original article is still online, as well as an updated version). Fowler explained how projects such as mine could maintain source code integrity with one simple build command. He also introduced a tool called CruiseControl that automatically maintained code integrity as it was built, and even used email to hold those who had broken the code accountable. While this may not seem like much now, with the advent of DevOps and associated tooling, back then it was really the only product of its kind and it went on to have a major impact on large and small development projects.

However, as I used Cruise Control and other tools like Jenkins and Bamboo, I found it odd that although we were using a collaborative build tool, configuration wasn’t given the same merit as source code. Why weren’t those changes tracked in the same way and for the same reasons? As I gained more experience with organizations that maintained internal ecosystems with hundreds of build and deployment projects, I found that the proliferation of duplication and boilerplate setups across projects was a productivity drain in and of itself. Initiatives such as GitHub’s scripts to rule them all  can help to minimize the work, but the work still exists, particularly for initial setup and ongoing maintenance.

But help was at hand. During a recent talk with BC Holmes, one of Intelliware’s leaders, she told me about two new build tools, LambdaCD and Concourse. When I researched them myself, I found that they facilitated script based management of the build pipeline as opposed to web based forms that force manual entry. This provided versioning, testability, refactoring and reuse.  Furthermore, the web interfaces provided are focused on emergent visualizations from the configuration and convenient manual commands.

ThoughtWorks Technology Radar shows some indication that web-based configurations of build pipelines may be a thing of the past. Just recently, they put a “Hold” on Jenkins, and followed that up with adding an “Adopt” on “Pipelines as code”.



It's only fair to share...
Share on FacebookGoogle+Tweet about this on TwitterShare on LinkedIn

Leave a Reply