The build and release process is the backbone infrastructure of software development. While it may not be the coolest or hottest part of the software development process, it is a very necessary one.
Why, you might ask?
Software delivery is about reduction of risk and waste, ROI on new features, and flow of value to our customers. Yes, we have heard these definitions before, but how do these definitions really relate to our everyday lives?
I would like to provide an analogy that might help explain why a software delivery pipeline is important and why the incorrect infrastructure (bad tools and practices) can create a negative impact on our software delivery.
We can think of the build and deployment process like a power plant pipeline (provided by solar, wind, gas, oil etc.). The source of the natural raw material (requirements) is captured from a set of smaller stations (developers) and then deposited into a centralized station (source control). It can then be packaged and regulated for distribution to other sub-stations.
The sub-stations further test and regulate the distribution and frequency (builds, and deployments) to other local-stations (QA and Staging) and then deliver to your home or office (production).
Now imagine, that you needed to deliver power from a power station (environment) using old technology and infrastructure. As the demands increase from either a heat wave or a cold snap, the pipeline and delivery system begins to fail, causing brown (rolling outages) and black (complete outages) outages. The station and pipelines that connect the entire grid (requirements to production) together, may not be designed to handle the increase of power and demand that the customers are asking for. The below picture tells a story of an ad-hoc pipeline that may not deliver the reliable, repeatable or reusable infrastructure we need. These type of ad-hoc pipelines are also impossible to scale or to accommodate the company and customers’ requests.
As the delivery of services fail, the customer’s trust of a dependable infrastructure quickly deteriorates. The cost to fix the infrastructure increases along with the cost of damage control and non-productive efforts.
In extreme events, like a complete collapse of a delivery system, the “Firefighting” or “Triage” efforts to keep the system functional in some fashion take us all away from productive initiatives and tasks.
Personally, I can think of any better way to spend our time and effort. For example, spending efforts focused on preventing and improving the delivery system as opposed to taking a “patch and pray” approach and waiting for the next event to happen.
My question when I see these older build and deployment infrastructures that are either built on 10 year old technology, or have been put into place ad-hoc with no real architecture or planning is, “Why haven’t you invested the time and resources to address and really fix the software delivery infrastructure that cause constraints and cost?”
The number one answer to my question is, “We don’t have time to fix it.”
The number two answers is, “The build and deployment system is not a priority, we can build it on our laptop and copy the files over to production.”
Both of these replies are very poor excuses!
From my experience, teams and individuals soon forget the impact of the last event and the cost to the customer loyalty. What’s worse is that the team never gets out of these cycles of firefighting and triage mode, creating a vacuum within the teams’ efforts that are hard to measure.
In addition, I have been at places were the software delivery pipeline does not get the visibility, investment and resources to create a new or improve the current system; to handle peak loads, or to deliver the power/features to the customer in a more efficient and effective manner.
If the delivery pipeline is not developed and designed correctly, then risk, constraints and complexes are introduced, which translate into additional cost and loss opportunities.
By having a software delivery system that is functioning at peak performance and can handle peak periods (the holiday season for example), we are able to deliver features to our customers at the speed and quality we need to.
The software delivery pipeline may not be the most impressive part of the software development cycle, but it is the critical infrastructure and grid that delivers the results and enables all of us to keep the lights on!