Thirty steps in, the agent is optimizing a goal you did not set

Re-anchor the goal at intervals through a long plan, and the agent finishes on the objective you gave it. Let it run uninterrupted and it drifts, step by reasonable step, into a goal it invented.

Balagei G Nagarajan3 min read

An agent's goal drifting over a long multi-step plan into something it invented

Key facts.

  • Natural Plan shows frontier models struggle to hold a realistic multi-step plan to completion, so long-horizon goal fidelity is not a given. source
  • Premise Order Matters shows reasoning is sensitive to how information is ordered, so an accumulating, reordering context can shift the agent's working goal. source

Why does the goal drift over a long plan?

Each step of a long plan involves a small reinterpretation of what to do next given the state so far. Individually these are reasonable; cumulatively they move. The agent optimizes the immediate subtask, then the next and the original objective recedes into an old entry under a growing pile of recent context. Natural Plan shows models already struggle to carry a multi-step plan to its end, and Premise Order Matters shows reasoning shifts with how information is ordered, so a long workflow that keeps reordering and accumulating context is a recipe for the agent's working goal to drift from the one you set. The drift is invisible step to step and obvious only in hindsight, when the finished work solves a goal nobody asked for.

The countermeasure is re-anchoring. At intervals through the plan, restate the original goal explicitly, compare the current trajectory against it and correct or escalate if they have diverged. This is the planning analog of re-reading the brief partway through a long project and it is precisely what an agent will not do unprompted. The re-anchoring turns an unbounded drift into a series of bounded segments, each realigned to the goal, so the agent that completes a thirty-step plan is still solving the objective you gave it rather than the mutated version it wandered into around step twelve.

A goal-fidelity curve over a long plan flattened by periodic re-anchoring

How do you hold the goal?

PracticeRun uninterruptedRe-anchored plan
The goalRecedes under contextRestated at intervals
TrajectoryNever compared to goalChecked against it
On divergenceDrifts furtherCorrected or escalated
At the endSolves a drifted goalSolves the real goal

Goal drift is structural: a frontier model held to step thirty optimizes a goal you never set and the rework lands late. (arXiv:2406.04520)

Re-anchoring needs a precise, stable statement of the goal to anchor to, which is what VibeModel provides as the Pattern Intelligence Layer. By making the success pattern explicit and persistent, it gives the agent a fixed reference to compare its trajectory against at each interval, so a long plan stays aligned to the objective you set instead of drifting into one the agent improvised across thirty steps.

Frequently asked questions

Why does a long plan drift?
Small per-step reinterpretations accumulate and models struggle to hold a multi-step plan to completion, so the original goal recedes under recent context.

What is re-anchoring?
Restating the original goal at intervals, comparing the agent's trajectory to it and correcting on divergence, the planning analog of re-reading the brief.

How often should you re-anchor?
Often enough that drift cannot compound far between checks, tuned to the plan's length and how consequential it is.


Share

Join the discussion

Have a take, a war story, or a question? Sign in with GitHub to comment and react. Comments are powered by GitHub Discussions, ad-free and yours to moderate.

Continue Reading

Every article here documents a failure mode we've seen in production. Faultmap finds them before you ship.