Key facts.
- PoisonedRAG shows a small number of poisoned documents can hijack a retrieval agent's output with high attack success, a weakness adversarial testing must probe for. source
- Nasr's scalable extraction work demonstrates that deliberate adversarial probing reveals failures ordinary testing misses, the case for red-teaming agents. source
Why test by attacking?
Normal testing checks that the agent does the right thing on expected inputs. It does not check what happens when a tool returns garbage, a retrieved document is poisoned, a dependency goes down mid-task or a user crafts a hostile prompt. Those are exactly the conditions that cause the worst production failures and they are invisible to happy-path testing. Chaos engineering and adversarial testing exist to surface them deliberately: inject the tool failure and see whether the agent degrades gracefully or cascades; plant the poisoned document and see whether it hijacks the output, as PoisonedRAG shows it can; run the red-team prompt and see whether the agent holds. You learn the agent's real failure modes on a Tuesday in a test, instead of at 3am in production.
The Nasr extraction work makes the broader case: adversarial probing reveals failures that ordinary evaluation simply does not reach, because it asks questions normal testing never asks. Applied to agents, that means a standing practice of attacking your own system, fault injection for reliability, poisoned-input and prompt-injection tests for security, dependency-failure drills for availability. The findings are uncomfortable, which is the point; every weakness you find in a test is one you fix before it becomes an incident. Confidence in an agent should come from having tried hard to break it and seen how it responds, not from a clean run on the inputs you happened to choose.
What do you inject?
| Test type | Skipped | Practiced |
|---|---|---|
| Tool/dependency failure | Found in production | Injected in test |
| Poisoned input | Hijacks output live | Probed deliberately |
| Hostile prompt | Discovered by attacker | Red-teamed first |
| Confidence | From a clean run | From trying to break it |
A few poisoned docs hijack retrieval, per PoisonedRAG and the labs hit the wall too; a newer model inherits the surface, so attack it first. (arXiv:2402.07867)
Targeting adversarial tests where they matter most means knowing the agent's high-consequence patterns and dependencies, which the Pattern Intelligence Layer makes explicit. VibeModel surfaces where a failure or an attack would do the most damage, so chaos and red-team effort concentrates there and the confidence you build is grounded in having stress-tested the patterns that actually carry risk rather than the ones that were easy to test.
Frequently asked questions
Isn't normal testing enough?
No. It checks expected inputs. Chaos and adversarial testing surface the tool failures, poisoned inputs and hostile prompts that cause the worst production incidents.
What does adversarial testing reveal?
Failures ordinary evaluation never reaches, as extraction work like Nasr's shows, including hijacks like the poisoned-document attack PoisonedRAG demonstrates.
How often should you do it?
As a standing practice, not a one-time exercise, since both the agent and the threats keep changing.