What the DORA Metrics Actually Measure, and Why Payments Teams Ignore Them · Midcore Operations
Midcore Operations
Cloud & Infrastructure

What the DORA Metrics Actually Measure, and Why Payments Teams Ignore Them

DORA's own research found that elite teams deploy more often and fail less, not one at the cost of the other. Most payments teams assume the opposite.

Published
22 October 2026
Reading time
9 min read
Author
Cloud & Infrastructure practice
Topic
Cloud & Infrastructure
Key takeaways
  • DORA, the DevOps Research and Assessment program now run by Google Cloud, measures software delivery performance across two dimensions: throughput (change lead time, deployment frequency, failed deployment recovery time) and instability (change fail rate, deployment rework rate).
  • The framework has evolved from its original four keys to a five-metric model, including replacing mean time to recovery with failed deployment recovery time and adding deployment rework rate, a detail most secondary explainers of DORA get wrong or skip.
  • DORA's research explicitly frames throughput and instability as two separate dimensions, not a tradeoff. The highest-performing teams in its research score well on both at once, deploying frequently while failing less often, not deploying slowly in exchange for fewer failures.
  • NIST's own configuration management guidance states that a change control board's formality should scale to a system's size, scope, and criticality, not apply the same heavy process to every change regardless of risk, which undercuts the assumption that compliance itself requires slow, infrequent deployments.

What the DORA metrics actually measure is software delivery performance across two separate dimensions, and most payments teams ignore them because they have the relationship between those two dimensions backwards. DORA, the DevOps Research and Assessment program now run by Google Cloud, measures throughput and instability as independent factors, not opposite ends of a single tradeoff. Its own research found that the best-performing teams score well on both at once. A payments team that treats slow, infrequent deployment as the safer choice is acting on an assumption the research behind these metrics does not support.

What DORA actually measures, and how the framework changed

DORA’s current model organizes its metrics into two groups. Throughput measures how many changes move through a delivery system over a given period. It uses three factors. Change lead time is the amount of time it takes for a change to go from committed to version control to deployed in production. Deployment frequency is the number of deployments over a given period. Failed deployment recovery time is the time it takes to recover from a deployment that fails and requires immediate intervention. Instability measures how well those deployments actually go, using two factors. Change fail rate is the ratio of deployments that require immediate intervention. Deployment rework rate is the ratio of deployments that are unplanned and happen as a result of a production incident.

That five-metric structure is itself a detail many explanations of DORA get wrong, because the framework has changed. DORA’s own site is explicit that these metrics have evolved alongside the technology landscape. They shifted from an original four-key model to the current five-metric one. Mean time to recovery, the metric most people still associate with DORA, has been replaced with failed deployment recovery time, a more precise measure specific to deployment failures rather than all incidents. Deployment rework rate was added as a genuinely new fifth metric. A payments team citing “the four keys” today is citing a version of the framework DORA itself has already moved past.

Throughput and instability are not a tradeoff

The single most important thing DORA’s research says about these metrics is also the thing most payments teams get backwards. Throughput and instability are presented as two separate dimensions of performance. They are not two ends of a dial where moving toward one necessarily moves away from the other. DORA’s research consistently finds that the highest-performing teams score well on both simultaneously. They deploy frequently and recover quickly. They also experience fewer failed deployments and less unplanned rework, not as a matter of luck but as a product of how their delivery pipeline is actually built.

This directly contradicts the mental model most payments organizations operate under, which treats deployment frequency and stability as competing goals to be balanced against each other. A team that reduces its deployment frequency specifically to reduce risk, believing that fewer changes means fewer chances to fail, is optimizing against a tradeoff the underlying research says does not exist in the form they are assuming. The actual lever DORA’s research points to is not deployment frequency itself. It is the quality of the pipeline, the testing, and the automation around each individual change. That quality is what determines whether higher throughput comes paired with higher or lower instability.

Why payments teams assume the opposite anyway

The belief that slow and infrequent deployment is inherently safer in a payments environment comes from a reasonable-sounding but incomplete idea. A payments system handles real money, so changes to it should be reviewed carefully. That idea then quietly gets translated into a second claim: changes should happen rarely, and only after a long approval process. The first half of that reasoning is correct. Changes to a payments system should absolutely be reviewed. The second half, that careful review requires infrequent deployment, does not actually follow from the first. This is exactly where the DORA research and the common payments-industry assumption diverge sharply.

A team that batches changes into large, infrequent releases to accommodate a slow review process is not making each change safer. It is making each release larger. A larger release concentrates more untested interactions into a single deployment, which typically increases, rather than decreases, the chance that something in that release causes a failure requiring intervention. The review process intended to reduce risk ends up amplifying it, by forcing the exact batching pattern that produces larger, riskier deployments instead of smaller, more easily verified ones.

What change management guidance actually says about formality

NIST’s own guidance on configuration management addresses this directly, in a context that applies well beyond payments. It describes the Configuration Control Board, the group responsible for reviewing and approving changes to a system. It states plainly that this board may be less formal for systems with limited size, scope, and criticality. The organization determines the size and formality of the board that is appropriate for a given system. Nothing requires applying one fixed level of ceremony to every change regardless of its actual risk.

That guidance undercuts the assumption that compliance itself requires a slow, heavy approval process for every deployment. A control that changes are reviewed, tested, and approved before reaching production can be satisfied by an automated pipeline that runs tests and requires a sign-off in minutes, just as validly as by a monthly change advisory board meeting. The requirement is that the control exists and functions. Nothing in that requirement specifies that it has to be slow, and NIST’s own guidance explicitly says the formality should scale to the actual risk of what is changing.

A concrete example: the monthly release that made things worse

Picture a payments platform that moved to monthly deployments years ago, specifically to give its change review board enough lead time to approve every release carefully. The intention was caution. The actual effect, over time, was that each monthly release grew to include dozens of unrelated changes, because any change that missed the monthly cutoff had to wait another month, which created constant pressure to cram in just one more fix before the window closed.

The result is a release with far more moving parts than any single change would have had on its own, reviewed by a board that cannot meaningfully evaluate the interaction between forty unrelated changes in the time allotted for one meeting. When something breaks after deployment, which happens more often than the team expects given how much “review” each release supposedly received, the failure is harder to isolate too, because so many things changed at once. The team’s change fail rate and deployment rework rate, the two instability metrics DORA tracks, are almost certainly worse than they would be with smaller, more frequent releases, even though the stated goal of the monthly cadence was exactly the opposite.

Where the actual risk control belongs

None of this means every change should ship instantly with no review at all. It means the review has to happen at the level of the individual change, where it can actually catch something specific, rather than at the level of a monthly batch, where forty changes get one combined sign-off nobody can meaningfully evaluate. A pipeline that runs automated tests against every single change, before it merges, is applying far more actual scrutiny per change than a human reviewer skimming a release note once a month ever could.

This is also where the instability metrics earn their keep as a feedback signal rather than a scorecard. A rising change fail rate is a specific, actionable signal that something in the pipeline, not the deployment cadence itself, needs attention: a gap in test coverage, a flaky environment, a step in the release process nobody has actually verified in months. Treating that signal as a reason to slow down further addresses the symptom by making every future release larger and harder to verify, which is precisely the pattern that produced the rising failure rate in the first place.

What measuring these metrics would actually show

A payments team that has never tracked its own change lead time, deployment frequency, failed deployment recovery time, change fail rate, and deployment rework rate is operating on an untested assumption. It has never actually checked whether its own tradeoff between speed and safety is real. Measuring these five metrics directly, rather than relying on the generic belief that slower is safer, is the only way to find out whether a given team’s actual instability problem is caused by deploying too often. DORA’s research suggests a different, far more common cause: deploying in large, infrequent, poorly tested batches that make every release riskier than it needs to be.

Building the pipeline automation, testing discipline, and deployment practices that let a payments platform increase throughput without increasing instability is exactly the work behind Midcore’s DevOps practice. Where the gap is specifically in the testing and release-validation discipline that makes frequent deployment safe rather than reckless, quality assurance and continuous delivery work is built to close it. And for a team that wants someone already measuring and acting on these metrics day to day, rather than discovering the problem the way the monthly-release team above eventually did, CloudOps is exactly that ongoing function.

Frequently Asked Questions

What are the actual DORA metrics, and has the framework changed recently?

DORA, the DevOps Research and Assessment program run by Google Cloud, currently measures five metrics across two dimensions. Throughput covers change lead time, deployment frequency, and failed deployment recovery time. Instability covers change fail rate and deployment rework rate. The framework evolved from an earlier four-metric model, replacing mean time to recovery with failed deployment recovery time and adding deployment rework rate as a fifth metric, a change most secondary explanations of DORA have not caught up with.

Do faster deployments actually mean more failures in payments systems?

DORA’s own research says no. It explicitly treats throughput and instability as two separate dimensions rather than a tradeoff, and finds that the highest-performing teams score well on both: deploying frequently while also failing less often. The assumption that slower deployment is inherently safer is not supported by the research the metrics are built on.

Does PCI compliance actually require slow, infrequent deployments?

No. NIST’s configuration management guidance states that a change control board’s formality should scale to a system’s size, scope, and criticality rather than applying the same heavy process to every change. Compliance requires that changes are reviewed, tested, and approved, which can happen through a fast, automated pipeline just as validly as through a slow manual board, depending on how the control is actually built.

Sources: DORA, DORA’s software delivery performance metrics, a research program run by Google Cloud, and NIST Special Publication 800-128, Guide for Security-Focused Configuration Management of Information Systems.

Cloud & Infrastructure practiceMidcore Operations · 22 October 2026
All insights
Free consultation

Get your free Assess

Tell us what is happening. We reply within one business day with a US-based practice lead, not a salesperson.

  1. 01Within 1 business dayA US-based practice lead replies and books a 30 minute call.
  2. 02On the callWe map the problem, the volume, the partners, and the constraints.
  3. 03After the callYou get a written read of one to three pages, yours to keep.
  • Handled under NDA
  • Written, not a slide deck
  • No obligation to continue

No obligation. We will tell you if you do not need us.

Before you go

Take the written read with you

Every engagement opens with an Assess: a written read of what is happening, what it costs, and what to do about it. It is quoted on its own and you can stop after it.

+1 (786) 619-0152