Key facts.
- The Amazon Q extension payload drove the agent to construct and run AWS CLI commands that listed and deleted resources (The Register).
- When agents build commands from text, the boundary between data and code disappears, the same root cause as SQL and shell injection; untrusted retrieved or constructed text can carry a hidden instruction, demonstrated against GPT-4 (Greshake et al., 2023).
- A stronger model is not the fix; on GAIA, GPT-4 with plugins solves only about 15% of tasks where humans reach 92% (GAIA, 2023).
Why not just tell the agent to be careful with inputs?
Once an agent builds a command from retrieved text, that text runs; a newer model obeys the hidden instruction, an incident demonstrated against GPT-4. (arXiv:2302.12173)
Because an instruction to be careful is not a parameterized query. Decades of injection defense taught one lesson: you do not sanitize your way out, you separate code from data structurally. Parameterized calls, allow-listed actions, and typed arguments keep untrusted text in the data slot where it cannot become a command. An agent that string-builds a command from retrieved text has reopened the exact wound the industry spent twenty years closing.
String-built vs. parameterized calls
| String-built call | Parameterized call |
|---|---|
| Untrusted text becomes command syntax | Untrusted text stays a data value |
| Any action the syntax allows | Only allow-listed actions |
| Injection runs as a command | Injection stays inert data |
VibeModel's Pattern Intelligence Layer recognizes when a constructed tool call departs from its normal shape, an unexpected flag, a command pattern that matches a destructive action, and stops it before execution. You parameterize and allow-list the calls; we catch the constructed command that does not match the pattern you intended. Structure first, pattern detection second.
Frequently asked questions
Does this apply to API calls too?
Yes. Any constructed call qualifies: shell, SQL, HTTP. If untrusted text shapes the call, it is an injection surface.
Can the model self-check its own command?
Unreliably. The same model that was injected is a poor judge of the injection. Enforce structure outside the model.