Key facts.
- MAKER demonstrates that long multi-step tasks only stay reliable when an explicit per-step error-correction layer catches the compounding mistakes a model makes on its own. source
- The NIST AI RMF's Manage function calls for documented, enforceable controls over AI behavior, which policy-as-code makes concrete and testable. source
- ISO/IEC 42001:2023 requires documented, enforceable controls over AI behavior, a guarantee a deterministic policy gate provides and a prompt instruction does not. source
Why isn't a rule in the system prompt enough?
A rule living only in the prompt gets reasoned around; a more capable model does not fix it, while an engine checks it every time (MAKER). (arXiv:2511.09030)
Because a rule in the prompt is a probabilistic request and a production policy needs to be a guarantee. The model reads "never issue a refund over $500 without approval" and follows it most of the time, but most of the time is not the bar for a financial control. Under an unusual phrasing, a long context, an adversarial input or just the model's own variance, it will eventually take the action the rule forbade and explain why it was justified. The rule was advice and the model treated it as advice. Policy-as-code makes the same rule a gate: the refund call is intercepted, the amount and approval status are checked by code and the action is blocked unless the conditions hold. The model never gets the chance to reason past it.
MAKER is the structural reason this matters. It shows that over a long chain of steps, small per-step error rates compound into near-certain failure unless something outside the model catches each error as it happens. A policy engine is exactly that something for the rules you cannot afford to break. A more capable model lowers the per-step error rate but does not make it zero, so over enough actions a probabilistic rule is broken eventually, while a deterministic gate is enforced on every single call. The agents that hold their hard limits in production are the ones where the hard limits live in code, not in a paragraph the model is asked to remember.
What belongs in the rules engine versus the prompt?
The split is by consequence. Soft preferences, tone, default behavior, the order to try things, belong in the prompt, where the model's judgment is the feature. Hard limits, spend caps, irreversible-action approvals, data-access boundaries, scope restrictions, belong in the engine, where determinism is the feature. The mistake teams make is putting a hard limit in the prompt because it was faster to write there and discovering after an incident that the limit was never actually enforced. The rule of thumb: if breaking the rule would be a security, financial or compliance event, it does not belong in the prompt alone.
| Rule type | Lives in | Why |
|---|---|---|
| Tone, defaults, ordering | Prompt | Model judgment is the value |
| Spend caps, approval gates | Policy engine | Must hold every time, deterministically |
| Data-access boundaries | Policy engine | A breach is not an acceptable miss rate |
The Pattern Intelligence Layer is where policy-as-code is expressed and enforced, so the hard limits a production agent must never cross are checked deterministically at the pattern level rather than requested in a prompt. Spend caps, approval gates and access boundaries become rules the model proposes against and the layer decides on. Reliability at the pattern level is what turns a policy from advice the model usually follows into a guarantee it cannot break.
Frequently asked questions
Can't a strong model just follow the rules in its prompt?
Most of the time. But "most of the time" is not the bar for a financial or security control and over enough actions the model will reason past a prompt rule eventually.
Doesn't a rules engine make the agent rigid?
Only on the hard limits, which should be rigid. Soft preferences stay in the prompt where the model's judgment is the point.
How do I decide what to encode?
By consequence. If breaking the rule is a security, financial or compliance event, it belongs in the engine, not the prompt alone.