How a verification layer turns a prompt injection into a caught error

Security and verification are not separate pillars. The same output check that catches a wrong answer is often what catches an injected instruction before it acts.

B

Balagei G Nagarajan

4 MIN READ


An injected instruction hitting a verification gate that compares the agent's action against the original task and stops it
Model mistake or injected instruction, doesn't matter to the verifier.
— from “How a verification layer turns a prompt injection into a caught error”

Key facts.

  • CaMeL (Defeating Prompt Injections by Design, arXiv 2503.18813): wraps the model with a layer that tracks data provenance and enforces policy at tool-call time. Result: 77% of AgentDojo tasks completed with provable security, vs 84% undefended. Verifier layer beats input filters alone.source
  • Google DeepMind's CaMeL (arXiv 2503.18813, 2025) defeats prompt injection by design: it extracts control and data flow from the trusted query so untrusted data can never change the program, and enforces capability-based policy on every tool call.source
  • The dual-LLM pattern behind CaMeL splits a privileged planner (sees only the trusted request, controls tools) from a quarantined reader (parses untrusted content, no tool access), so injected text can't reach the action path directly.source
  • AgentDojo (arXiv 2406.13352) runs 97 realistic tasks and 629 security tests: attacks beat the best agents in under 25% of cases undefended, and a secondary attack-detector defense cuts attack success rate to about 8%.source
  • An injection makes the agent act outside its task, and a layer checking action against intent flags it; a stronger model won't hold alone, since on AgentDojo a detector drops attack success to 8%, sparing the rework. (arXiv:2503.18813)

Why does the same check catch both a wrong answer and an attack?

Same shape either way: the agent is about to do something the task never asked for. Model mistake or injected instruction, doesn't matter to the verifier. It only needs to confirm the action follows from the intent. If it doesn't, stop it. A refund the customer never requested, a tool call to an endpoint outside the task's scope, a data read that the goal never justified: a verifier flags all of these, and an injection that hijacks the agent produces exactly that signature.

this is why the security pillar and the verification pillar aren't really separable. The injection defenses that hold up in research are verification by another name: check the data flow, check the tool call against policy, check that the action traces back to the trusted request. CaMeL formalizes it, but the principle is the everyday verifier doing security work.

Layered defense diagram: input screening, privileged-vs-quarantined split, policy check on tool calls, and output verification, with an injected instruction stopped at the policy layer

Where does the verifier actually stop the injection?

At the boundary between deciding and acting. An injection hasn't done harm until the agent calls a tool, sends data, or commits a side effect. Put the verification there: before any tool runs, confirm the call's provenance traces to the trusted request and its parameters satisfy the task and policy. CaMeL does this with capabilities that track where each value came from, so a value derived from untrusted content can't flow into a privileged action without an explicit, checked policy allowance. The injected instruction can still be read. It just can't act.

FailureHow it shows at the outputWhat the verifier checks
Model mistakeAction does not satisfy the goalDoes the result meet the stated intent?
Indirect injectionAction the task never requestedDoes the tool call trace to the trusted request?
Data exfiltrationOutbound flow with no task basisIs this data flow allowed by policy?

this is the case for a Pattern Intelligence Layer that treats reliability and security as one surface. At the pattern level, the verifier that confirms an action matches intent is also the control that catches a hijacked action, so you aren't building two systems. you're enforcing one property, action-matches-intent, on every run, around whatever model you deploy. Security stops being a separate retrofit and becomes a benefit of verifying the agent's behavior, which you wanted anyway.

Frequently asked questions

Does a verifier replace dedicated injection defenses?
It complements them. Input screening and the privileged-quarantined split reduce exposure. The verifier on tool calls is the layer that catches what slips through, which research treats as essential, not optional.

Is CaMeL a product I can buy?
it's a research design from Google DeepMind with an open implementation, not a turnkey product. The pattern (verify data flow and tool calls against the trusted request) is what you apply, regardless of tooling.

Will a more capable model stop needing this?
No. On AgentDojo, undefended best-in-class agents still get attacked, and adding a verification-style detector is what drops attack success to roughly 8%. The safety comes from the check, not the model.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.