If you have attended an industry event or conference lately, the talk inevitably drifts to development approaches. One thing you hear about a fair amount is the difficulties in adoption of automated testing, continuous integration, continuous delivery CI/CD pipeline, Agile testing, DevOps, Test automation, Behavior Driven Development, Continuous Testing among others. And while these may sound like buzzwords to the uninitiated, organizations today are adopting many of these approaches successfully to drive faster release cycles along with high quality software.
A Statista report shows that only nine percent of tech professionals surveyed had not adopted DevOps in any which way. A large majority had either fully embraced DevOps or well on its way to the transition.
If your organization is new to DevOps or in the midst of implementing a CI/CD pipeline, then there are some common mistakes and pitfalls that you can avoid during the transition.
Above all, a change of this degree is not merely a technical shift but a process and cultural re-alignment. Here are some of the most common mistakes related to culture, teams, tools and processes that you can avoid:
1. Culture and Attitude Shift
If you have noticed the DevOps infinity loop diagram, then you may have seen that there is no specific time for testing in it. This is because in the DevOps way, testing is a continuous activity, a fundamental part of every single output.
With CI/CD, testing and quality assurance are integral to everything the developers do. Now, this requires a shift in the mindset to include testing as an integral part of each task.
Testing becomes something that all team members are doing all the time as part of their routine activities.
This shared ownership for quality doesn’t happen easily. It is tough to break old patterns and you need to facilitate this however you can.
Not Holding Retrospectives
DevOps depends on the relay of constant feedback. Continuous improvement is not possible unless you make room for collaboration and communication.
A company that is not organizing retrospective meetings will find it tough to implement this culture of continuous feedback or CI/CD. While retrospective meetings are a Scrum/Agile fundamental, they are equally necessary to DevOps. This is because a retrospective meeting or a “retro” inculcates the habit of holding meetings to exchange feedback. In fact, one of the very items in an initial retro meeting is to set up recurring retro meetings so it becomes a ritual.
When it comes to software quality, all team members share ownership of it. For instance, the developers can write unit tests and also write the code with testability in mind, helping to mitigate risk from the start.
One simple way to help reflect the change in the view of testing is to change the testers’ titles from QA to software tester or better yet, quality engineer. While this change may seem surface-level, too simple, or even silly, if someone has the title of “software quality assurance,” it conveys the wrong message about who has the responsibility to ensure the product quality (which in Agile, CI/CD and DevOps is everyone).
Another fundamental aspect is to be aligned in terms of quality, what quality means for the team, organization, stakeholders, and for each team member.
2. The Definition of Done
If quality is a continuous and shared process, then there needs to be a shared understanding of the definition of done. What constitutes as done? What transpires when an item is marked as done on the Trello board or Kanban board.
This Definition of Done (DoD) is quite a powerful tool in the DevOps/CI CD context. It helps create a wider understanding of the quality standards of what the team builds and how.
DoD improves project visibility and facilitates the CI/CD process as long as you have a transparent and mutually agreed upon DoD.
3. Not setting realistic, well-defined goals
This is one of the most oft-quoted advice but it bears repeating. For the success of any major change such as CI/CD or DevOps, one needs to set tangible goals and measure performance against them. What is it that you are trying to achieve with CI/CD? Is it reducing the release time with better quality.
Any goals set should not only be transparent and realistic, they should be adjusted to the company’s current context. For instance, how frequently do your customers need new fixes or versions?
There’s no need to over engineer processes and release faster if there is no added customer experience benefit.
Also, you don’t always need to implement both CD and CI. For instance, highly regulated sectors like banks (BFSI) and healthcare companies can work just fine with CI.
CI serves as a good starting point for any company adopting DevOps. It creates a fundamental shift in the way organizations deliver software. Once they have mastered CI, it is time to think of process improvement, increasing release velocity if needed, and other improvements.
For many organizations, CI alone will suffice and CD must only be implemented if it brings additional value.
4. Lack of relevant dashboards and metrics
Once your goals are established, the scrum team can build a dashboard to measure the KPIs. It is best to implement a progressive assessment before you start designing a dashboard.
Various reports and gadgets are useful for different team members. A scrum master is more interested in status and coverage. While senior leadership might be interested in the burndown rate.
Some teams also use a traffic light dashboard for CI/CD to indicate if they are on track or off. Red means you really need to take care of your priorities.
However, if the dashboards aren’t well-defined they can be misleading too. Analyse what data everyone needs and then establish a standardized narrative for what the data points mean. Find out what makes more sense to the stakeholders, graphs, text or number.
5. No manual testing
Test Automation lays the groundwork for a good CI/CD pipeline. But extensive automated testing doesn’t mean you pay zero attention to manual testing. Do you need specific CI testing tool?
To build an effective CI/CD pipeline, you will need a fair bit of manual testing too. There are always going to be some aspects of testing that need that human insight and analysis.
Instead, think about integrating your manual testing efforts into the pipeline. Once the manual testing of certain test cases is completed, you can move into the deployment phase.
6. Not building a culture of better testingAn effective CI/CD pipeline needs organizational buy-in, access to the right kind of tools be it test management or integration and continuous measurement.
Building a strong, quality-centric culture focuses on implementing tests, monitoring customer experience after deployment and tracking the improvements. For this, here are a few practical tips that you can easily implement:
- Ensure that tests are easy to write and flexible enough not to break when you refactor your code
- Product teams must be invested in the testing process — listing user stories that are important to validate during CI pipelines
- While you can’t have complete test coverage, always ensure that the most important flows for UX and customer experience are always covered.
Last but not least
The transition to CI/CD is typically triggered from the bottom up but at the end of the day it is a transformation that needs management buy-in. CI/CD is all about skillset, processes, tools and culture shift that require attention from the management in terms of time and resources.