Thirty steps in, the agent is optimizing a goal you did not set

Re-anchor the goal at intervals through a long plan, and the agent finishes on the objective you gave it. Let it run uninterrupted and it drifts, step by reasonable step, into a goal it invented.

B

Balagei G Nagarajan

3 MIN READ


An agent's goal drifting over a long multi-step plan into something it invented

Key facts.

  • Natural Plan shows frontier models struggle to hold a realistic multi-step plan to completion, so long-horizon goal fidelity is not a given. source
  • Premise Order Matters shows reasoning is sensitive to how information is ordered, so an accumulating, reordering context can shift the agent's working goal. source

Why does the goal drift over a long plan?

Each step of a long plan involves a small reinterpretation of what to do next given the state so far. Individually these are reasonable; cumulatively they move. The agent optimizes the immediate subtask, then the next and the original objective recedes into an old entry under a growing pile of recent context. Natural Plan shows models already struggle to carry a multi-step plan to its end, and Premise Order Matters shows reasoning shifts with how information is ordered, so a long workflow that keeps reordering and accumulating context is a recipe for the agent's working goal to drift from the one you set. The drift is invisible step to step and obvious only in hindsight, when the finished work solves a goal nobody asked for.

The countermeasure is re-anchoring. At intervals through the plan, restate the original goal explicitly, compare the current trajectory against it and correct or escalate if they have diverged. This is the planning analog of re-reading the brief partway through a long project and it is precisely what an agent will not do unprompted. The re-anchoring turns an unbounded drift into a series of bounded segments, each realigned to the goal, so the agent that completes a thirty-step plan is still solving the objective you gave it rather than the mutated version it wandered into around step twelve.

A goal-fidelity curve over a long plan flattened by periodic re-anchoring

How do you hold the goal?

PracticeRun uninterruptedRe-anchored plan
The goalRecedes under contextRestated at intervals
TrajectoryNever compared to goalChecked against it
On divergenceDrifts furtherCorrected or escalated
At the endSolves a drifted goalSolves the real goal

Goal drift is structural: a frontier model held to step thirty optimizes a goal you never set and the rework lands late. (arXiv:2406.04520)

Re-anchoring needs a precise, stable statement of the goal to anchor to, which is what VibeModel provides as the Pattern Intelligence Layer. By making the success pattern explicit and persistent, it gives the agent a fixed reference to compare its trajectory against at each interval, so a long plan stays aligned to the objective you set instead of drifting into one the agent improvised across thirty steps.

Frequently asked questions

Why does a long plan drift?
Small per-step reinterpretations accumulate and models struggle to hold a multi-step plan to completion, so the original goal recedes under recent context.

What is re-anchoring?
Restating the original goal at intervals, comparing the agent's trajectory to it and correcting on divergence, the planning analog of re-reading the brief.

How often should you re-anchor?
Often enough that drift cannot compound far between checks, tuned to the plan's length and how consequential it is.


Share this post

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Find where your agent breaks, before you build it

Faultmap maps where your agent will fail from the goal and your data, then hands you the first test suite it has to pass.