Key facts.
- Greshake demonstrated indirect prompt injection, where an agent obeys instructions hidden in data it retrieves, a failure that becomes a compliance event in a regulated workflow. source
- DORA (EU Regulation 2022/2554) imposes ICT risk-management, resilience-testing and incident-reporting obligations on financial entities, requirements an ungoverned agent does not meet. source
- In Moffatt v. Air Canada (2024 BCCRT 149), a tribunal held a company liable for its chatbot's wrong answer, establishing that an agent's output is the company's legal position. source
Why is the risk higher now than a year ago?
Because the rules an agent has to satisfy are arriving and tightening and an agent deployed without governance falls further behind each one. DORA sets concrete obligations on financial entities, ICT risk management, resilience testing, mandatory incident reporting and those are not aspirational; they are requirements with enforcement behind them. An agent already in production with no audit trail and no oversight model does not become compliant on its own when the rule takes effect; it becomes a known gap that someone now has to close under time pressure. The Moffatt decision shows the other edge: a tribunal already held a company responsible for what its chatbot told a customer, so the legal exposure does not wait for a dedicated AI statute, it is here through ordinary liability.
A more capable model does not shrink this. The Greshake result is why: an agent that reads a document and follows instructions hidden inside it will take an action nobody authorized and a stronger model still does this when the injection is well-constructed. In a regulated process, that action is a compliance failure and without the records and oversight the regulation expects, you cannot show what happened, who approved it or that you had controls in place. The business risk is not abstract. It is the difference between an incident you can explain and defend and one you cannot and the governance you build now is what determines which one you are holding when the question comes.
What does the exposure actually cost?
It shows up in three forms. Regulatory: a high-risk agent without the required logging and oversight is a finding waiting to be made, with remediation on the regulator's timeline, not yours. Legal: as Moffatt shows, the agent's output is your position, so a wrong answer in a binding context is a liability you own. Operational: retrofitting governance into a live agent is harder and slower than building it in, because you are now changing a running system under scrutiny. All three are smaller, often far smaller, if the governance was there from the start.
| Exposure | Ungoverned agent | Governed agent |
|---|---|---|
| Regulatory | Known gap, remediate under pressure | Records and oversight already in place |
| Legal | Indefensible wrong action | Reconstructible, attributable decision |
| Operational | Retrofit a live system under scrutiny | Controls built in, no scramble |
The Pattern Intelligence Layer is where the records, oversight and boundaries the regulations expect are enforced, so an agent meets the rising bar by construction rather than by retrofit. Lifetime logging, human-oversight gates and scope boundaries are tracked at the pattern level, which is exactly what a regulator or a plaintiff asks you to produce. Reliability at the pattern level is what keeps an agent deployment defensible as the rules keep arriving.
Frequently asked questions
Can't we add governance when the regulation actually applies?
You can, but retrofitting a live agent under a regulator's timeline is far more expensive than building it in. The gap is known the moment the rule lands.
Is the legal risk real without an AI-specific law?
Yes. Moffatt v. Air Canada held a company liable for its chatbot's answer under ordinary liability. The agent's output is already your legal position.
Does a better model lower this exposure?
No. The requirements are legal and procedural. A stronger model still takes unauthorized actions under injection and without records you cannot defend them.