Key facts.
- AgentDojo, the prompt-injection benchmark, makes the gap concrete: GPT-4o stays useful on benign tasks yet still falls to a targeted injection roughly 48% of the time, which is exactly the per-tool scoping, injection validation, and audit-trail testing a demo skips. source
- Reported: security is the top barrier to deploying autonomous agents in production, with surveys citing it as the leading blocker for a majority of enterprises. source
- A defensible production agent uses a layered architecture: input validation, runtime filters, permission scoping to limit blast radius, and output validation with structured audit logging. Demos ship none of these layers. source
- NIST's AI RMF 1.0 (January 2023) frames accountability, traceability, and lifecycle risk management as production requirements, none of which a sandbox demo exercises. source
- A demo runs with full permissions and no attacker, setting a bar production cannot meet, and a stronger model does not patch the gap, since adaptive attacks broke 12 defenses. (arXiv:2406.13352)
What exactly does the demo leave out?
Start with the environment. A demo gives the agent one user's mock data and full access, so there are no competing trust boundaries to get wrong. Production hands the agent shared corporate resources, third-party inputs it did not author, and multiple access tiers. That turns the agent into a classic confused deputy: real authority, and an attacker able to supply the intent. The demo never had to solve that because the demo never had an attacker.
Then the controls. Permission scoping does not matter in a sandbox because there is nothing sensitive to over-reach into. Audit trails do not matter because no one will investigate a demo. Injection defenses do not matter because the inputs are friendly. Identity controls do not matter because there is one user. Every control that defines an enterprise-ready agent is invisible in the demo precisely because the demo removed the conditions that make the control necessary. The result is a believable but false impression that the hard part is the capability, when the hard part is everything the demo took off the table.
How do you close the gap before it bites?
Treat the demo as a capability proof, never a security blueprint. Before production, scope every tool to least privilege, give the agent its own identity instead of a user's session, validate untrusted inputs and treat ingested content as hostile by default, and capture an audit trail you could hand to an incident responder. These are the layers NIST and adversarial benchmarks like AgentDojo point at, and they are the difference between an agent that demos and an agent that ships.
| Condition | Consumer demo | Enterprise production |
|---|---|---|
| Permissions | Full, single user | Scoped per tool, least privilege |
| Inputs | Friendly, authored | Untrusted, third-party |
| Identity | One user's session | Distinct agent identity |
| Audit | None | Reconstructable trail required |
| Attacker | Absent | Assumed present |
This is the work a Pattern Intelligence Layer does by default. Reliability and security at the pattern level mean scoping, identity, untrusted-input handling, and audit are properties of how the agent runs in production, enforced on every run rather than assumed away by a demo. The capability the demo showed still holds. The controls it hid are now in place, and they stay in place when you upgrade the model underneath.
Frequently asked questions
So is the demo useless?
No. A demo proves capability, which is real and useful. It just proves nothing about security, because it ran in conditions that removed every security risk. Use it as a capability proof, not a readiness signal.
What is the single biggest demo-to-prod jump?
Trust boundaries. The demo had one user and full access; production has shared resources, untrusted inputs, and multiple tiers. That shift is what turns an agent into a confused deputy.
Will a frontier model make production safe by itself?
No. Adaptive attacks defeat published defenses against frontier agents. Security has to live in the controls around the agent, which is exactly what the demo left out.