Pavamana AI LabsThe Hard 70%

Your agent is only as trustworthy as the worst source it reads

A poisoned wiki page, a planted document, a tampered tool response. Agents inherit the trust you placed in their data, and attackers know it.

B

Balagei G Nagarajan

3 MIN READ

Multiple data sources feeding an agent, one of them subtly contaminated

Key facts.

  • PoisonedRAG: five injected documents reached a 90% attack success rate, via Wikipedia edits, fake news, malicious sites, or an insider (PoisonedRAG, USENIX Security 2025).
  • The Amazon Q extension was poisoned at the source through a malicious pull request that reached an official release (SC Media).
  • Tool responses are sources too: a compromised or spoofed API can feed an agent attacker-controlled data it treats as ground truth.

Why doesn't the agent notice the source is bad?

Because nothing tells it to doubt. The agent treats a retrieved document, a repo file, or an API response as authoritative, and it has no built-in notion of provenance or trust level. A poisoned source looks identical to a legitimate one. The defense is to attach trust to sources explicitly: weight by provenance, verify critical facts across independent sources, and treat any externally writable source as untrusted by default. The agent will not invent that skepticism; you have to give it.

Heatmap of data sources scored by trust level with one writable external source flagged red

Flat trust vs. provenance-aware

Flat trustProvenance-aware
All sources weighted equallySources weighted by trust level
One poisoned source drives the actionCritical facts cross-checked
Writable sources trustedExternally writable sources tainted

Source poisoning is cheap and model-agnostic: PoisonedRAG hit 90% with five texts, and a more capable model reads the poison just as faithfully. (arXiv:2402.07867)

VibeModel's Pattern Intelligence Layer watches for the pattern of a source steering an agent toward an action that does not match its history, the signature of a poisoned input, and flags it. You attach provenance and cross-check critical facts; we catch the moment a single source starts driving the outcome. Trust your data deliberately, not by default.

Frequently asked questions

How do I find five poisoned documents in millions?
You usually cannot by inspection. Defend by provenance and cross-source verification of the facts that drive actions, not by hunting the needle.

Are internal sources safe?
Only if they are not externally writable. A wiki or repo anyone can edit is an external source wearing an internal badge.

Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Security

Five poisoned documents can hijack your whole RAG

The model is not compromised. The knowledge base is. A handful of planted documents, crafted to rank first for a target question, steer the agent to a chosen wrong answer with a citation.

4 min read

Security

Untrusted text becomes a tool argument, and that's the breach

Your agent reads a support ticket, a web page, a GitHub issue, and turns the words into tool calls. If those words carry instructions, the agent constructs the attacker's parameters and runs them.

5 min read

Security

How one poisoned message turns a coordinated agent team against itself

In a multi-agent system, the message from one agent is untrusted content to the next. An attacker only has to poison one link.

3 min read

Security

The channel between your agents is an attack surface too

You secured the front door. The agents talk to each other through a hallway you left unwatched, and that is where the next breach walks in.

3 min read

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.

See how Faultmap worksCompare vs your eval stack