Key facts.
- ConflictBank (arXiv:2408.12076) evaluated LLMs on scenarios with conflicting information between context and parametric knowledge; models frequently failed to recognize conflicts and update reasoning accordingly, instead anchoring on their prior belief (ConflictBank, arXiv:2408.12076).
- MINJA (arXiv:2503.03704) demonstrated that LLMs asked to update a multi-step plan based on new information often accept the stated update without correctly propagating its implications to all affected downstream steps.
- Position bias in LLM reasoning (arXiv:2406.07791) shows that information presented early in a context window has disproportionate influence on LLM outputs, meaning tool responses that contradict an early plan often fail to override the original plan.
- In agentic contexts, "stale plan execution" - where an agent executes a plan developed with outdated or incorrect information even after receiving contradicting tool output - is a documented source of task failure that does not manifest as an obvious error.
- Explicit plan revision steps, where the agent pauses after each tool call to check whether the plan needs updating before proceeding, are rare in production agent architectures because they add latency and most tool calls confirm rather than contradict the plan.
Why agents proceed despite contradicting evidence
The ConflictBank finding points at a structural tendency in LLM reasoning: models are optimized to be helpful and to move forward. When tool output partially contradicts a plan, the path of least resistance is to interpret the contradiction as minor and proceed. This is often the right call. When it is not, the error is silent: the agent completes the workflow, the result is wrong, and the trace shows a sequence of logical-seeming steps that led to a wrong conclusion.
Position bias compounds this. If the plan was established in the first 10% of the context window and the contradicting tool output arrives later, the model's generation is influenced more by the early plan than by the late contradiction. The contradiction is "seen" by the model but does not override the prior plan because the prior plan has more positional weight. This is a known LLM architectural characteristic, not a reasoning failure, and it requires an architectural response rather than a better model.
The plan revision architecture pattern
The pattern that addresses this adds an explicit plan revision step after each tool call that returns substantive information. The revision step asks: does this output change any assumption in the current plan? If yes, which steps need to be modified before we proceed? This step can be implemented as a model call with a focused prompt (what changes?), as a structured check against the plan's key assumptions, or as a deterministic comparison of expected values versus returned values. The key is that it is explicit and happens before the next step executes.
Conflict types and recommended responses
| Conflict type | Example | Default agent behavior | Correct behavior |
|---|---|---|---|
| Value conflict | Planned for 100 units, tool returns 50 available | Proceed with original quantity (wrong) | Revise plan for available quantity |
| Availability conflict | Assumed record exists, tool returns not found | Attempt to proceed, fails at next step | Halt and revise before next step |
| Constraint conflict | Planned action, tool returns unauthorized | Retry or proceed with escalation | Update plan to exclude prohibited action |
| Temporal conflict | Planned for future state, tool returns current state | Use future state assumption (stale) | Use returned current state |
VibeModel's Pattern Intelligence Layer identifies which tool calls in your specific deployment are most often followed by task failures that a plan revision would have prevented. By analyzing the correlation between tool output values and subsequent agent reasoning errors, it identifies which tool calls carry high conflict risk and should trigger explicit plan revision checks. That prioritization makes the architectural investment tractable: add plan revision gates at the tool calls where they matter most, rather than adding overhead to every step.
Frequently asked questions
Can a stronger model learn to revise its plan on its own?
ConflictBank shows models keep the old plan when output contradicts it; a more capable model ignores it too. (arXiv:2408.12076)
Does adding a plan revision step at every tool call add prohibitive latency?
Not if scoped correctly. A focused "does this change the plan?" prompt with a one-sentence answer adds 100-300ms per step and one model call. For workflows where incorrect plan execution costs hours of remediation, that latency is easily justified. For high-frequency, low-stakes workflows, a lightweight value comparison check can replace the model call entirely.
How do we write an effective plan revision prompt?
Frame it as a specific binary check: "The plan assumed X. The tool returned Y. Does this require changing step N or any subsequent step? Answer yes or no, then explain." The specificity prevents the model from providing an elaborate justification for proceeding unchanged when a plan revision is actually warranted.
Can position bias be mitigated by placing the current plan after the most recent tool output?
Partially. Placing the most recent tool output near the end of the context window, where it is more proximate to the model's generation point, reduces but does not eliminate position bias. Combined with an explicit conflict check, the combination is more reliable than either approach alone.