Agents bring risk categories your application and data governance never had to name

Traditional app-sec and data governance assume the software does what it was told. An autonomous agent can be told something new by the data it reads. That gap needs its own controls.

B

Balagei G Nagarajan

4 MIN READ


An iceberg with familiar app and data risks above the waterline and agent-specific risks hidden below
Excessive agency: the agent can take actions broader than its task needs.
— from “Agents bring risk categories your application and data governance never had to name”

Key facts.

  • The OWASP GenAI Security Project published a Top 10 for agentic applications, cataloguing agent-specific risks, excessive agency, tool misuse, identity and memory abuse, that traditional app-sec lists do not cover. source
  • The Cloud Security Alliance has published agentic AI security guidance, including threat modeling and identity frameworks, treating autonomous action as a new control domain. source
  • Greshake and colleagues showed indirect prompt injection, where retrieved content becomes instructions the agent follows, a category with no equivalent in software that only executes its own code. source

Why don't traditional controls cover agent risk?

Because traditional application and data governance assume the software's behavior is fixed at build time. You review the code, you control who can call it, you classify the data it touches and the program does what it was written to do. An agent breaks that assumption. Its next action is chosen at run time from its context and its context includes data it reads from the world, which means an attacker who controls a document the agent retrieves can change what the agent does. Greshake's indirect prompt injection is the defining case: there is no analogue in software that only runs its own instructions, so there is no traditional control for it. OWASP built an agentic Top 10 because the existing LLM and app-sec lists did not name excessive agency, tool misuse or memory poisoning and the Cloud Security Alliance added agentic threat modeling for the same reason.

A more capable model expands these categories rather than closing them. A more autonomous agent has more agency to be excessive with, more tools to misuse and more reach for an injection to exploit. Injection mitigations lower the success rate but do not zero it, so the category persists no matter how good the model is. The point is not that agents are ungovernable; it is that governing them means recognizing risks that classic frameworks were never built to see and putting controls under each one: bounded autonomy, scoped tools, validated inputs and an agent identity you can hold accountable.

Tree diagram branching traditional risks on one side and agent-specific risk categories on the other

What are the agent-native risk categories?

Excessive agency: the agent can take actions broader than its task needs. Instruction injection: untrusted content it reads steers its behavior. Tool misuse: it invokes capabilities in unintended ways or combinations. Identity and memory abuse: its credentials or its retained state are corrupted or borrowed. Goal misalignment: it pursues a literal objective in a way that violates intent. Each one needs a control that traditional governance does not supply, which is why OWASP and CSA built agent-specific guidance instead of extending the old lists.

Agent-native riskWhy traditional controls miss itControl to add
Excessive agencyCode review assumes fixed behaviorBounded autonomy
Instruction injectionNo analogue in fixed-code softwareInput trust boundaries
Tool misusePermissions assume known call patternsScoped, monitored tools
Identity / memory abuseStatic identity modelAccountable agent identity

The Pattern Intelligence Layer is where the agent-native risks get named and controlled. Excessive agency, injection exposure, tool misuse and identity abuse are tracked at the pattern level, so each gets a control instead of falling through the gap between application and data governance. Reliability at the pattern level is how an enterprise governs the risks that only autonomous software creates.

Frequently asked questions

Will a stronger model retire the agent-specific risks?
Injection and unbounded tool use are a new risk class: a more capable agent sharpens them, injection survives mitigations and skipping the controls is rework. (arXiv:2302.12173)

Can't we extend our existing app-sec program to agents?
Partly, but it misses the agent-native categories: excessive agency, injection through read content, tool misuse. OWASP and CSA built dedicated agentic guidance because the old lists do not name them.

Why is instruction injection a new category?
Because classic software runs only its own code. An agent chooses its next action from data it reads, so untrusted content can become an instruction, which has no traditional equivalent.

Does a safer model remove these risks?
No. A more autonomous agent has more agency and reach and injection mitigations lower but do not zero the rate. The categories persist; the controls are what manage them.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.