Key facts.
- METR's RCT found experienced developers 19% slower with early-2025 AI tools while estimating they were about 20% faster, across 246 real tasks. source
- The slowdown came partly from time spent reviewing, correcting and reworking AI-generated output, a cost the productivity narrative omits. source
- Because nearly half of AI-generated code carries a vulnerability, the verification is not optional, it is what keeps insecure changes out. source
Why does verification cost so much?
Because reviewing code you did not write, to a standard high enough to ship, is genuinely hard work and there is a lot of it. The agent produces a change quickly, but to put that change into production you have to understand what it does, confirm it is correct, check it is secure and make sure it does not break something elsewhere and the agent's fluent confidence does not reduce any of that effort. The METR result quantifies the trap: skilled developers, measuring carefully, were slower with AI than without and sincerely believed the reverse. The generation felt like a speedup, so the review felt like overhead that should not be necessary and that gap between felt and actual is exactly where teams underbudget. They priced the fast generation and forgot the slow verification, then wondered why the project did not get faster.
The reason you pay it anyway is that the alternative is worse. Veracode's finding that 45% of generated code is insecure means unverified agent output is a steady stream of vulnerabilities, so the review is not a tax on a working process, it is the process. Skip it and you trade review time for production incidents, which cost far more.
How do you budget for it honestly?
Count verification as part of the cost of using the agent and measure the real cycle time, not the felt one. Automate what you can, security scanning, tests, static analysis, so the human review focuses on what machines cannot check. Then judge the agent on end-to-end time including verification, against the baseline of doing it the old way, because METR shows the felt speedup will mislead you. An agent earns its place when the total time, generation plus verification, beats the alternative, not when the generation alone looks fast.
| What you budget | What actually happens |
|---|---|
| Generation time only | Verification cost surprises you, slows the cycle |
| Generation plus verification | Honest cycle time, insecure changes caught |
METR found devs 19% slower with AI while feeling faster, the gap eaten by review; a newer model writes quicker, verifying stays slow. (arXiv:2507.09089)
Making verification efficient and visible is part of what VibeModel does as the Pattern Intelligence Layer. We model the patterns that a change must satisfy and automate the checks, so the verification cost is real but bounded and the agent is judged on the total time it actually takes.
Frequently asked questions
Does AI never speed developers up?
It can, but METR shows the perceived gain is unreliable and verification often erases it. Measure total cycle time, not feel.
Can I automate the verification away?
Partly. Scanning and tests catch much of it; human review handles what they cannot. The cost shrinks but does not vanish.
Why pay the review cost at all?
Because 45% of generated code is insecure. The review is what keeps those vulnerabilities out of production.