The DevOps methodology has been around for a few years now and has recently become the “in thing’ for organisations to implement. Just as there was a major push to “cloudify” everything, organisations are now looking to introduce DevOps in all things IT, and why should not they, considering the benefits that can be realised. Organisations are becoming larger, with a global presence making business processes complex. Organisations have also understood the importance of the data they collect and hold relying on data-driven decision-making tools to grow, relying on automation to accelerate this process. DevOps practices helps by giving incremental enhancements to further accelerate this process through automated provisioning, continuous integration & deployment, automated testing leading to product-oriented teams resulting in a structural organisational change.
The DevOps Ecosystem
As organisations head down the DevOps path, one of the key things that need to be considered is the choice of the framework and platform to architect development, integration, deployment, and monitoring processes along with the tools and cultural shift to be incorporated into these processes. The introduction of these DevOps processes and tools into the software/application development lifecycle aims to create a speed boost to each phase and therefore the overall process. The DevOps processes, tools, frameworks, platforms, and shift in culture together from the DevOps Ecosystem. Although the DevOps Ecosystem can be customised to meet each organisations’ individual needs, there are some key tools that need to be a part of the ecosystem to make this work.
- Source Code Management: It is critical that all code, be it application code or platform orchestration scripts be under source control. Tools like GitHub, Bitbucket, Subversion, Stash (Bitbucket Server) etc need to be set up and configured along with the processes to ensure their use.
- Automated Build: Once code is checked into the SCM, the DevOps processes calls for the code to be built and packaged ready for testing in an automated manner. Tools like Maven, Apache Ant, Jenkins, SonarQube, Artifactory, etc. can help with this.
- Configuration Management: Ensuring consistency of environments is critical to the DevOps process. It stops the “Works on my PC” syndrome that is commonplace in organisations that delays deployment of new features that the business needs. Tools like Ansible, Chef, Puppet, Terraform, Vagrant etc. help maintain this consistency through an “Infrastructure as Code” model and deploying a known good configuration to all environments.
- Test automation: has been around for a while now, but the focus with DevOps is integrating test automation with the build phase to ensure that the code is always in a production deployable condition. Tools like Selenium, JUnit, Bamboo help orchestrate the automated testing as part of the build phase.
- Security & Monitoring: the final set of tools to close out the DevOps ecosystem is around ensuring reliable monitoring and security setup for your infrastructure as well as application to understand the impact of new features introduced through the DevOps pipeline and its impact when released into production. Tools such as CloudWatch, Prometheus, Nagios, Kibana, Logstash, Datadog, Splunk, Graphite etc. can help you navigate this.
Pitfalls to look out for …
Despite its growing popularity and broad adoption, the overarching concept of DevOps is still new to many organizations and things do not always go according to plan. Transformation projects, especially in large organisations can go horribly wrong, and often do go wrong. 4 of the most common reasons that DevOps initiatives fail are:
1. Not explicitly defining what DevOps means to the organisation
The word “DevOps” can have different meanings from one organisation to another, based on the choices they make and the value the organisation is looking to derive from it. For example, DevOps can mean vastly different things for organisations making airplanes when compared with an innovative technology start-up. It can also mean different things to people within an organisation, with Developers equating it to a specific approach to build and deploying software, while Management viewing it as a continuation of existing process improvement initiatives with an emphasis on faster time-to-market on new features. Before starting any DevOps transformation journey, it is critical that the organisation agree on what DevOps means for the organisation, and the business value that introducing DevOps will bring.
2. Focusing purely on tools and techniques
One of the most common mistakes that I see organisations make is elevating technology as the primary driver for DevOps Transformations at the expense of processes and the culture change needed to deliver quality outcomes for their customers. Merely hiring release engineers, giving access to on-demand infrastructure is not enough.
For any successful DevOps initiative, the people and processes must be considered. There is no benefit in increasing the flow of features, only to be bottlenecked at the Quality Assurance or Release Management stages. The impact on downstream teams must be identified for the DevOps transformation initiative to be successful.
3. Ignoring Governance
The need for DevOps in a lot of enterprises usually starts as a rebellion against a slow, cumbersome IT Organisation that takes weeks or months to release new features into production. These teams, working outside the centralised IT structures were free to innovate and develop systems and processes to enable faster release cycles. While this approach may work in smaller organisations or start-ups, it is not effective in an enterprise setting.
Multiple independent teams trying to release new features into production at the same time can be a recipe for disaster. DevOps is not about reducing the number of governance gates. On the contrary, it enables more effective, more frequent governance gates to shine a light on complex release orchestration challenges.
4. Not collecting Metrics
Another mistake many organisations make after committing to the DevOps methodology, is not understanding how well the DevOps transformation is working if it works at all!
The bottom-line is that if you are using the DevOps methodology, you need metrics, and you need good metrics tools. Without metrics, you will not have any way of knowing if your DevOps implementation is doing what you want it to do, or if it includes problem areas that need your attention. Most DevOps metrics fall into three broad categories: People, Process, and Technology.
At the very least, 4 key metrics that need attention are: Deployment/Change Frequency, Change Lead Time, Change Failure Rate, and Mean Time to Recover (MTTR).
In Concussion
A lot of organisations make the mistake of thinking that getting the correct tools in place to create a workable CI/CD pipeline is enough to reap the benefits of DevOps within their organisations. This could not be further from the truth as the getting the tooling and processes in place is only half the battle. The other half which is the more difficult half is developing a “DevOps Culture” and having it ingrained in every side of how the organisation runs. To successfully introduce and implement DevOps, the development team, the operations team, and the business must work together and not in their individual silos. The shift in mindset needed to achieve this is large and can only happen in increments, with incremental benefits seen along the journey. The subject of introducing this cultural change is another challenge and I plan on writing about it soon.