Key facts.
- DORA (EU Regulation 2022/2554) requires financial entities to set and monitor measurable ICT resilience objectives and report against them, governance bound to defined criteria.source
- Bain's Technology Report 2025 found it is tough to prove generative AI's value without clear KPIs, so teams without defined metrics struggle to distinguish a healthy agent from a degrading one.source
- MIT NANDA's "State of AI in Business 2025" tied measurable business impact, not activity, to the small share of pilots that succeeded, underlining the need for outcome KPIs (reported).source
Why does governance need success criteria to work?
Capability does not say what this agent is for: without KPIs governance has nothing to measure and a more capable model's gains never surface either (Bain). (source)
Governance makes decisions. Decisions need evidence. If you haven't defined what the agent is for and what good looks like, the governance review has nothing to judge against. It becomes a status meeting. Reports activity, not outcomes. Defined KPIs give governance a yardstick. Is the agent delivering what it was built for? At what quality, what cost, within what boundaries? Bain's finding backs this up. Teams without defined metrics couldn't tell a healthy deployment from a degrading one. Their governance, however well-attended, couldn't actually govern.
A better model won't write your success criteria. Only you know what this agent is for. MIT NANDA's 2025 report found the pilots that worked were tied to measurable business impact, not activity. What isn't measured against a target can't be judged. DORA makes this mandatory for financial entities: measurable resilience objectives, monitored and reported. Governance that names KPIs up front and reviews against them can actually do its job. Keep the agent, change it, retire it, decided on evidence, not impression.
What KPIs should governance hold an agent to?
Four dimensions. Outcome: is it delivering the business result it was built for? Quality: correct enough, at an acceptable error rate? Cost: within target per outcome? Boundary: staying inside its defined scope? Each gets a target at launch and a measured value in review. The review's job is to act on the gap. Improve, re-scope, or retire, not just note it. KPIs without a review that acts on them are dashboards. A review without KPIs is theater. Together, they're governance.
| KPI dimension | Question it answers | Governance action on a gap |
|---|---|---|
| Outcome | Delivering the result? | Improve or retire |
| Quality | Correct enough? | Add controls or re-scope |
| Cost | Within cost per outcome? | Optimize or downgrade |
| Boundary | Inside its scope? | Tighten or alert |
The Pattern Intelligence Layer is where success criteria become a governance instrument. Outcome, quality, cost and boundary KPIs are tracked at the pattern level and tied to a review that acts on them. Governance decides on evidence whether an agent stays, changes or goes. Reliability at the pattern level is what turns governance from a ritual into a function that actually judges the agent.
Frequently asked questions
Isn't a governance review enough on its own?
No. Without success criteria it has nothing to judge against and becomes a status meeting. KPIs give the review the yardstick that lets it decide.
What KPIs matter most for an agent?
Outcome, quality, cost per outcome and boundary adherence. Together they tell governance whether the agent is delivering, drifting or due for retirement.
Why can't the model define success for us?
Because success is about your business purpose, not the model's behavior. Only you can say what the agent is for and what outcome justifies keeping it.