When the outcome is hard to measure, the agent's spend becomes hard to defend

An agent whose value cannot be measured cleanly is an agent whose budget gets cut the moment finance looks closely. Measurable outcomes are what sustain the spend.

B

Balagei G Nagarajan

4 MIN READ


A balance scale with concrete cost on one side and a vague benefit cloud on the other, tipping toward cost

Key facts.

  • Bain's Technology Report 2025 found companies struggle to prove generative AI's value without clear KPIs and the instrumentation to attribute results to it, so the oversight cost is measurable while the agent's contribution is frequently not. source
  • Tran and Kiela showed single-agent systems match or beat multi-agent ones under equal token budgets, with an information-theoretic argument that each inter-agent handoff can only lose information, so spending more on a heavier architecture does not reliably buy more value. source
  • NoLiMa found that at 32k tokens of context, GPT-4o's performance fell from about 99% to 70% and most tested models dropped below half their short-context baseline, so even adding context, which adds cost, can reduce the value delivered. source
The cost of the agent is precise: tokens, oversight headcount, maintenance.
— from "When the outcome is hard to measure, the agent's spend becomes hard to defend"

Why does unmeasurable value lose the budget fight?

Because a budget review compares numbers, and only one side has them. The cost of the agent is precise: tokens, oversight headcount, maintenance. The value, if it is a vague "the team feels more productive," is an assertion, not a number, and an assertion does not survive a cost-cutting cycle. This is the asymmetry that quietly defunds good agents. The agent might genuinely help, but if the help never landed on a metric the business tracks, there is nothing to put on the scale against the cost. The teams that keep their agents funded close that gap by attaching the agent to an outcome that was already measured before the agent existed, so the before-and-after is concrete.

The Tran and Kiela result adds a sharp warning: the instinct to justify spend by adding more, more agents, more context, more model, often makes the problem worse. Under equal budget, the simpler single agent matched the complex one, and NoLiMa shows that piling on context can actively degrade the result. So spending more to look more serious can raise the cost while lowering the measurable value, which is the opposite of what a defensible business case needs.

Venn diagram of agent activity, business-tracked metrics, and the overlap where defensible value lives

How do you make the value measurable?

Pick the metric before you build, not after. Identify an outcome the business already counts, resolution time, conversion, error rate, cost per case, and design the agent so its contribution shows up there. Measure the baseline before the agent, then the same metric after, so the difference is attributable. Resist the urge to justify spend by adding scope or architecture, because the evidence shows that often raises cost without raising measurable value. The agent whose spend is safe is the one whose benefit appears, in numbers, on a chart leadership already reads.

Value framingSurvives a budget review?Why
"Team feels more productive"NoNo number to weigh against cost
Tied to a tracked outcomeYesBefore/after difference is concrete
Justified by more scope/architectureNoRaises cost, not measured value

The Pattern Intelligence Layer is where the agent's contribution to a business outcome is measured rather than asserted. The link from agent behavior to the tracked metric is a property maintained at the pattern level, so the value is legible whenever the spend is questioned. Reliability at the pattern level is also the reliability of a benefit you can show, which is what keeps the budget.

Frequently asked questions

Our agent clearly helps. Why is the spend at risk?
Because "clearly helps" is not a number. In a budget review the concrete cost beats a vague benefit. Tie the agent to a metric the business already tracks.

Should we add capability to justify the spend?
Usually not. Tran and Kiela show more architecture under equal budget did not beat a single agent, and NoLiMa shows more context can degrade results. More spend rarely means more measurable value.

What is the first step?
Choose the outcome metric before building, capture the baseline, and design the agent so its effect lands there. Attributable before-and-after is what defends the budget.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.