Context fills with tool outputs, intermediate reasoning, retrieved documents, and prior steps.
<p><b>Key facts.</b></p>
<ul>
<li>Long context does not mean reliable context: on NoLiMa, at 32,000 tokens 11 of 13 leading long-context models fell below half their short-context score, and GPT-4o dropped from 99.3% to 69.7% (Adobe Research, <a href="https://arxiv.org/abs/2502.05167" target="_blank" rel="noopener">NoLiMa, arXiv:2502.05167</a>, ICML 2025).</li>
<li>Models keep getting worse the more you feed them: Chroma tested 18 frontier models and every one degraded as input grew, with a 200,000-token-window model showing real degradation by 50,000 tokens (Chroma, Context Rot, 2025).</li>
<li>The usable window is smaller than the advertised one: RULER puts effective context at roughly 50 to 65% of the stated length (<a href="https://arxiv.org/abs/2404.06654" target="_blank" rel="noopener">RULER, arXiv:2404.06654</a>, 2024).</li>
<li>Success collapses as tasks get longer: frontier models hit near-100% on tasks a human finishes in under 4 minutes and under 10% on tasks over about 4 hours (METR, <a href="https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/" target="_blank" rel="noopener">Measuring AI Ability to Complete Long Tasks</a>, 2025).</li>
</ul>
<h2>What does it mean for an agent to forget the goal?</h2>
<p>The agent does not forget the goal in a literal sense. The goal is still in the context window, usually near the top. What happens is that the model stops weighting it. Context fills with tool outputs, intermediate reasoning, retrieved documents, and prior steps. The objective sentence from step 1 is one line among thousands by step 20. Attention is finite. Recent, voluminous material crowds out the early instruction. Two visible results: distraction, where the agent drifts into locally useful tangents without reconnecting to the original goal; and goal substitution, where a sub-goal mentioned mid-run becomes more salient and the agent optimizes for that instead. The output is coherent. It solves the wrong problem.</p>
<h2>Why does a longer context make it worse, not better?</h2>
<p>In a short session the goal is recent. It sits close to the newest tokens. Attention weights it naturally. At 20 steps it has receded. Tool results and scratchpad fill the middle. The model reasons from what happened most recently, not from what was originally asked. More context means more material for recent content to crowd out early instructions. This is not a capability failure. The architecture makes it predictable.</p>
<h2>Why do long-horizon tasks fall apart?</h2>
<p>Long tasks accumulate more intermediate outputs, more tool results, more sub-task completions. Each one adds to the pile that buries the original goal. At 50 steps, the task specification from step 1 is a distant memory in the context. The model is running on recency. An agent optimizing for the last thing it did rather than the thing it was asked to do is not a bug in the model. It is a predictable consequence of how attention distributes across long contexts.</p>
<div class="fig"><img src="/blog/article15-diagram.png" alt="A curve of task accuracy falling as context length and number of steps grow, with the original goal token shown shrinking in attention weight from the start to the end of the run"/></div>
<h2>Why doesn't a 200K or 1M context window solve this?</h2>
<p>Longer context windows do not solve the attention distribution problem. They just give the model more to work through. The original instruction is still at position 0. With more content after it, it becomes even more diluted relative to the recent material. A 1M context window and a drifted goal is still a drifted goal. The fix is structural, not window size.</p>
<h2>What does goal drift look like in a running agent?</h2>
<p>The agent starts on the right task and gradually shifts. A sub-task gets completed and the agent moves to the next logical step, which was not part of the original ask. Or a retrieved document mentions a related topic and the agent follows it. Or the agent re-reads its own intermediate output and optimizes for making that more complete instead of returning to the original question. The trace is coherent throughout. The final output just answers something different than what was asked.</p>
<h2>How do you keep the agent on goal?</h2>
<table style="width:100%;border-collapse:collapse;margin:1.5rem 0;font-size:0.97rem;"><tr><th style="text-align:left;padding:10px 12px;border-bottom:2px solid #e5e7eb;color:#5E6AD2;font-weight:600;">Symptom</th><th style="text-align:left;padding:10px 12px;border-bottom:2px solid #e5e7eb;color:#5E6AD2;font-weight:600;">Cause</th><th style="text-align:left;padding:10px 12px;border-bottom:2px solid #e5e7eb;color:#5E6AD2;font-weight:600;">Fix</th></tr>
<tr><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Goal ignored mid-run</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Instruction buried under accumulated context</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Restate the goal each step, pin it at the end of the prompt</td></tr>
<tr><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Accuracy falls as run grows</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Context rot, attention dilution</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Summarize or drop old history, cap the prompt size</td></tr>
<tr><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Wanders to a related task</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Task derailment, no goal check</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Verify each step against the original objective</td></tr>
<tr><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Big window, still drifts</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Effective context below advertised</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Keep the working prompt under about half the window</td></tr>
<tr><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Loses tone or constraints</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Constraints de-prioritized over time</td><td style="padding:10px 12px;border-bottom:1px solid #eee;vertical-align:top;">Hold constraints in external state and re-inject them</td></tr>
</table>
<p>Restate the goal at each decision point. Keep a compact goal statement in a working memory slot that persists across steps, separate from the growing scratchpad. Before any action, compare the proposed step against the original objective. If the agent produces an output, have it check: does this address what was actually asked. How often that check is needed, and where in your specific workflow goals are most likely to drift, is what VibeModel maps as the Pattern Intelligence Layer.</p>
<h2>Frequently asked questions</h2>
Will a bigger context model hold the goal longer?
The goal gets buried as context grows; a better model defers the cost. (arXiv:2502.05167)
Isn't a bigger context window the fix?
No. RULER shows effective context is roughly half to two-thirds of the advertised size, and Chroma shows degradation well inside the window. A bigger window holds more, but it does not keep the goal salient; it usually dilutes it further. Prune and restate instead.
Why does the agent do fine on short tasks and fail on long ones?
Because length is the stressor. A short task has few steps and a small context, so the goal stays salient. METR shows success falling sharply as task length grows, since every extra step adds context that buries the objective and another chance to drift.
Is this the same as lost-in-the-middle?
Related. Lost-in-the-middle is why a goal placed mid-context gets under-weighted; goal forgetting is the agent-scale version, where a long trajectory pushes the original instruction into that low-attention zone and keeps it there.
What is the single highest-impact fix?
Re-inject the goal at the end of the context on every step and check the step against it. That keeps the objective where attention is strongest and catches drift before the agent has wandered far.