Key facts.
- AgentDojo (arXiv:2406.13352) is a benchmark of realistic tasks and prompt-injection test cases for tool-using LLM agents, showing that injected instructions can move agents off their intended policy. source
- The OWASP Top 10 for Agentic Applications (ASI categories) names prompt injection and excessive agency among the core agentic risk classes a control layer must address. source
- ISO/IEC 42001:2023 treats risk handling as implemented Annex A controls and operational safeguards, not as instructions the system may choose to honor. source
Why isn't an in-prompt policy enforcement?
Because the prompt asks the model to behave and asking is not the same as preventing. The model reads the boundary and, most of the time, respects it, which is enough to make the prompt look like a control in testing. But the model is the thing being governed and a control that lives inside the thing it governs can be overridden by anything that influences the model: an injected instruction in retrieved content, a cleverly framed user request or plain probabilistic drift on an edge case. AgentDojo exists to measure exactly this, putting tool-using agents in realistic tasks with adversarial injections and the result is that the agent can be moved off its policy. When that happens, the in-prompt rule does not stop the action, because the rule and the decision both live in the model the attacker just steered.
A stronger model raises the bar for the attacker without changing the category. It may resist more injections and drift less, but the boundary is still a request it can decline and the OWASP agentic risks, prompt injection and excessive agency among them, remain present because they are properties of letting a model decide. The enforceable version of a policy lives where the model cannot reach it: in the code that mediates the agent's tools and permissions, which checks the action against the rule and blocks it regardless of what the model decided.
What does enforced policy look like for an agent?
Controls that sit between the agent and its effects, not inside its reasoning. Permission scopes that the agent physically cannot exceed, so an over-reach is denied at the tool layer rather than trusted at the prompt layer. Action checks that validate what the agent is about to do against the policy before it happens, so a violating call is blocked even if the model was convinced to make it. Approval gates on irreversible or high-consequence actions, so a human confirms what the agent cannot be trusted to decide alone. And policy expressed as enforceable rules, policy-as-code, rather than prose the model interprets. The prompt can still carry the boundary as guidance, which helps the common case, but the enforcement is the code that does not negotiate. That separation is what turns a readable policy into a real one.
| Aspect | Policy in the prompt | Policy enforced in code |
|---|---|---|
| Nature | Request the model may decline | Check the model cannot bypass |
| Under injection | Can be overridden | Still blocks the action |
| Scope | Advisory | Permission and tool boundary |
| High-stakes actions | Trusted to the model | Gated, sometimes to a human |
A policy the agent only reads is a suggestion, and AgentDojo shows injection moves agents off task; a bigger model does not make the prompt a control, enforce it in code before the incident. (arXiv:2406.13352)
The Pattern Intelligence Layer is where enforcement meets observation. Policy boundaries and the actions that approach them are tracked at the pattern level, so attempted violations show up as signals and the enforced gate has the context to allow, block or escalate. Reliability at the pattern level is what makes a policy something the agent runs inside, not something it is merely told.
Frequently asked questions
Isn't a well-written system prompt enough for most cases?
It handles the common case but is not enforcement. AgentDojo shows injections can move agents off their instructions, so anything that matters needs a check in code that the model cannot override.
Where should the enforcement live?
At the tool and permission boundary, between the agent and its effects. Permission scopes, action checks and approval gates block a violating action regardless of what the model decided.
Does a stronger model let me skip code-level controls?
No. A stronger model resists more attacks but the boundary stays a request it can decline. The OWASP agentic risks persist, so the enforceable control belongs outside the model.