Without auditability, an agent cannot be trusted in a high-stakes role

In high-stakes work, trust is not a feeling. It is the ability to reconstruct what the agent did and why. Auditability is the prerequisite, and regulators are about to require it.

B

Balagei G Nagarajan

4 MIN READ


A reconstructible audit trail forming a bridge between an agent's action and the trust that lets it run
Auditability is not a reporting feature; it is the condition under which trust is even possible.
— from “Without auditability, an agent cannot be trusted in a high-stakes role”

Key facts.

  • ISO/IEC 42001:2023 requires documented information, records and internal audits for an AI management system, so an agent's operation can be traced and examined, making auditability a management-system prerequisite. source
  • DORA (EU Regulation 2022/2554) requires financial entities to maintain records and reporting that let ICT incidents be reconstructed and reviewed, the regulated form of auditability. source
  • On tau2-bench (arXiv:2506.07982), a dual-control benchmark, evaluated agents reached only around the mid-30s percent overall success across domains, so high-stakes actions need a reviewable record. source

Why is auditability the prerequisite for trust?

Because in high-stakes work, trust is operational, not emotional. You trust an agent to act in finance, healthcare or legal not because it feels reliable but because, when it acts, you can reconstruct exactly what it did, what information it had, what it decided and why and so you can catch an error, defend a decision or satisfy a regulator. Take that record away and you have an agent whose actions are unexaminable, which means they are undefendable and no serious organization extends trust to something it cannot examine after the fact. ISO/IEC 42001 makes this concrete by requiring documented records and internal audits for the AI management system, and DORA requires financial entities to keep records that let incidents be reconstructed. Auditability is not a reporting feature; it is the condition under which trust is even possible.

A more capable model makes the audit trail more necessary, not less, because it gets deployed into higher-stakes roles where the cost of an unreconstructable error is larger. tau2-bench, which tests agents in a setting where both the agent and a user act on shared state, found overall success only around the mid-30s percent across its domains, so a high-stakes agent will produce wrong actions at a rate that demands the ability to review each one. The agents that earn trust in regulated, high-consequence work are the ones whose every action leaves a record complete enough to answer "what happened and why," because that record is what a regulator will require and what your own organization needs to stand behind the agent at all.

Tree diagram decomposing an agent action into the captured record elements that make it reconstructible

What makes an agent action auditable?

A record that reconstructs the decision, not just logs the output. What inputs and context the agent had. What it retrieved and from where. What it decided and what reasoning led there. What action it took and what resulted. Linked together so an action can be traced end to end. That is the difference between a log that says the agent did X and a record that lets you understand why it did X and only the second one supports trust in a high-stakes setting or approval from a regulator.

Logged but not auditableAuditable
Final action recordedInputs, retrieval, decision, action, result
"What" without "why"Reasoning reconstructable
Fragments across systemsLinked end-to-end trail
Cannot defend a decisionDecision can be examined and defended

The Pattern Intelligence Layer is where auditability is built in rather than bolted on. The inputs, retrieval, decisions and actions that reconstruct an agent's behavior are captured at the pattern level, so every high-stakes action leaves a record a regulator or an auditor can examine. Reliability at the pattern level is what makes a high-stakes agent trustworthy enough to deploy and defensible enough to keep.

Frequently asked questions

Isn't logging the agent's outputs enough for an audit?
No. Logging "what" without "why" cannot reconstruct a decision. Auditability means capturing inputs, retrieval, reasoning, action and result, linked end to end.

Why do regulators care about auditability specifically?
Because a decision that cannot be reconstructed cannot be reviewed. ISO/IEC 42001 and DORA both require records that let an agent's operation be traced after the fact.

Does auditability matter if the agent is accurate?
Yes. tau2-bench shows even evaluated agents fail often, so high-stakes actions need a reviewable record. Accuracy does not remove the need to examine what happened.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.