Why DevSecOps Teams Struggle with Metrics (And What to Do About It)
Most DevSecOps teams are drowning in toolchain data but starving for insight. They have GitLab, Jenkins, SonarQube, Jira, and a dozen other tools all generating metrics. But when it comes time to answer basic questions like "Why did our deployment fail rate spike last month?" or "Are we actually getting faster?", the answers aren't there.
This isn't a technology problem. It's a strategy problem.
The Real Issue
The root cause is almost always the same: tooling accumulates organically as teams grow, but nobody stops to think about how the data fits together.
Platform engineering sets up CI/CD. Security adds vulnerability scanning. QA configures test reporting. Each team optimises for their own needs without considering the bigger picture.
The result? Data silos everywhere. Conflicting definitions of basic metrics. "Deployment frequency" means something different to the platform team than it does to engineering leadership. "Lead time" has three different definitions depending on who you ask.
What Actually Works
The teams that get this right share a few characteristics:
They start with questions, not data. Instead of asking "What metrics can we pull from our tools?", they ask "What decisions do we need to make?" The measurement strategy follows from there.
They invest in definitions. It sounds boring, but agreeing on what DORA metrics actually mean for your organisation is foundational. When everyone knows exactly how "deployment frequency" or "change failure rate" is calculated, the arguments stop and the insights start.
They accept that it takes time. Building a solid metrics foundation isn't a weekend project. It requires sustained attention over months. But the teams that make this investment stop firefighting data issues and start making better decisions.
Where to Start
If you recognise your team in this post, here's a practical starting point:
- Pick your three most important delivery metrics
- Write down exactly how each one should be calculated
- Check if everyone on the engineering leadership team agrees with those definitions
- If they don't (and they probably won't), you've found your first problem to solve
The good news is that this is fixable. The bad news is that it requires work. But the payoff, making decisions with confidence instead of gut feel, is worth it.