What shapes the long-term success of your CI/CD effort? A faster, more automated pipeline for software development? We recently outlined 4 success factors when getting started with CI/CD – and all led back to culture.
That’s helpful, but it’s also productive to look at the downsides – not of CI/CD itself, but of common mistakes organizations make, especially when they’re just starting out.
[ Need a primer on continuous integration/continuous delivery? Read What is CI/CD? ]
“If done right, proper continuous delivery and integration can lead to great customer experiences,” says Joy Scharmen, director of DevOps at Optimizely.
In the interest of doing CI/CD right, we asked Scharmen and other IT leaders to share some of the common ways teams go wrong. Here’s what they had to say.
1. Don't follow the recipe for tool salad
I could walk into a Home Depot today and buy a bunch of expensive tools and materials; I’d walk out still not knowing how to build a new back deck on my house. Thankfully, I have just enough common sense to not spend money loading up my garage with a bunch of stuff that I don’t really understand.
IT teams need to apply the same common sense when determining the right tools and technologies for their CI/CD pipelines, or they risk ending up with what Nate Berent-Spillson, senior director of delivery at Nexient, calls “tool salad.” (Mmm, sounds delicious.)
“I’ve seen companies go on a buying spree, picking up this tool, that platform, and that system over there, yet in the end, they weren’t equipped to build an iOS app,” Berent-Spillson says. “The unspoken idea was that if they bought enough tools, the capabilities they wanted would have to be in there somewhere.”
Choosing the right platforms and tools is indeed an important part of your CI/CD plan, but your needs should dictate the tools, not vice versa.
“Instead of thinking about what tools you need, think first about the capabilities you want, like ‘build pipeline for iOS,’ ‘build pipeline for Android,’ ‘mobile app distribution for both platforms,’ or 'signing certificate security and management,’” Berent-Spillson advises. “That way, you’re focusing on how to move value through the system. The choice of tools will follow from that.”
2. Don't mess up on metrics and feedback
Continuous measurement is an important success factor for CI/CD, but it comes with a downside: Picking the wrong things to measure.
“One of the biggest pitfalls to watch out for is picking the wrong metrics to test,” Scharmen at Optimizely says. “Businesses can easily guide their products down the wrong direction if they choose the wrong metrics to track and measure.”
Because CI/CD and DevOps often go hand-in-hand, there can be some overlap in the metrics you use to evaluate their impacts. The key principle here: Track what actually matters to your organization. As Gordon Haff, technology evangelist, Red Hat, wrote previously about DevOps metrics: “A metric is properly thought of as a key performance indicator for data – a measurement that’s important to you in some fundamental way.”
[ Read the full article: DevOps metrics: Are you measuring what matters? ]
The same idea applies to your CI/CD pipeline: Make sure you know what’s important, not just to you but to the organization.
“Different members of an IT organization or team have different priorities, so it’s important to assess which data everyone wants and needs from the start,” Scharmen says.
Scharmen notes another related pitfall: Getting duped by data via misinterpreting it, confirmation bias, or other reasons. This applies not only to your ongoing measurement but to testing itself in your pipeline.
“Misinterpreting a test can be worse than not running one at all and can quickly negate the value of an experimentation by injecting uncertainty and fear into a process that’s meant to provide clarity and confidence,” Scharmen says.
Finally, don’t forget about your actual production application or service. CI/CD doesn’t actually come with a defect-free guarantee as code moves into production. You still need a feedback loop there.
“Avoid the pitfall of having automated the delivery of software from source code to end user but not automating the collection of live, passive feedback, either from end users or automated tests,” says Maxwell Bloch, consultant at Applied Information Sciences (AIS). “For example, instrument your application to provide telemetry and error occurrences. Many bugs will be detected earlier in the CI/CD process, but some bugs will only occur with real usage.”
3. Don't give people a way to bypass the CI/CD pipeline
Don’t enable team members to bypass the pipeline other than under the most extraordinary of circumstances. This can be particularly important if you’re dealing with skepticism or cultural stasis in the earlier stages of your shift to CI/CD.
“If your CI/CD platform is not the only way, excluding an emergency break-glass method, to deploy code to your production environment, it will become tempting for teams to work around it when your testing [finds] code issues that are not easily solvable,” says Jerry Gamblin, principal security engineer at Kenna Security.
A promise of CI/CD is that it will enable application teams to move faster than before, but there may still be real reasons to slow down, and that can cause some frustration or feelings of failure. (This is another reason why cultural health is as important as technical decisions and architecture: A healthy culture can handle the speed bumps.)
You might automate your pipeline iteratively or in small chunks, and that’s fine. Similarly, you might build a CI/CD pipeline for a specific application team – also fine. But no matter your approach, don’t effectively give people the chance to opt out of your CI/CD pipeline if they decide it doesn’t suit them for some reason.
