Your agent is running a ten-step data processing task. It fetches records, enriches them from three APIs, validates them, transforms the format, and uploads the result. Each step takes thirty seconds.
At step seven — four and a half minutes in — the enrichment API returns a timeout.
The agent has no checkpoint. It doesn't know how to resume from step seven. The only option the framework gives it is to restart. So it restarts. It re-runs steps one through six, which it already completed successfully, then hits step seven again.
The API times out again.
It restarts again. This time, step three's API call fails because of rate limiting from the first two runs. The agent has now introduced a new failure it didn't have originally.
This is checkpoint blindness. It's not exotic. It's what happens by default in most agent frameworks when no one explicitly builds recovery logic.
Why this pattern is more damaging than it looks
The wasted work is the small problem. Re-running completed steps costs time and compute. That's annoying but measurable.
The side effects are the real problem. Steps one through six may have had real-world effects: records written to a database, API calls that trigger downstream systems, emails queued. When the agent restarts and re-runs those steps, those effects happen again.
Database writes get duplicated. Orders get placed twice. Emails get sent twice. API rate limits get burned twice. The re-run doesn't just waste time — it creates data integrity issues that may take hours to untangle.
The error accumulation problem. Each restart that burns API rate limits or modifies shared state changes the environment the agent will run in on the next attempt. The agent faces a harder problem on each retry than it did on the first attempt. Eventually it fails in a way it didn't fail originally.
What tasks need checkpointing
Any task that:
- Takes more than a minute to complete
- Makes writes or external calls at intermediate steps
- Has steps that depend on the output of earlier steps
- Could be interrupted by transient failures (network timeouts, API rate limits, OOM)
Which is most production agent tasks of any complexity.
What checkpointing looks like
A checkpoint is a record of state at a known-good point: what has been completed, what the output was, what comes next. When a failure occurs, the agent resumes from the last checkpoint rather than from the beginning.
This requires:
- Explicit step boundaries — the agent knows where each step begins and ends.
- State serialization — the output of each step is stored in a durable location (database, object storage) before the next step begins.
- Idempotent steps — re-running a completed step produces the same output without duplicate side effects. Either steps are naturally idempotent or they check whether they've already run before executing.
- Resume logic — when a failure is detected, the agent loads the most recent checkpoint and continues from there rather than restarting.
None of this is conceptually complex. All of it requires deliberate design decisions that most teams skip in the early builds.
The diagnostic before you build
Before you deploy any multi-step agent, answer these questions:
- If this agent fails at step N, what happens? Does it resume? Restart? Stop?
- Do any of its steps have side effects? Are those steps idempotent?
- What's the longest this agent might run? If it fails halfway through, how much work is lost?
- What happens if this agent runs twice against the same input?
If the answers are "it restarts," "yes but not idempotent," "thirty minutes," and "data corruption" — the checkpointing needs to exist before the agent ships.
Faultmap maps the failure surface of long-horizon tasks specifically. It identifies which steps have unprotected side effects, where checkpoints should be inserted, and whether the retry logic creates compounding failures. You see the recovery gaps before they become production incidents.
The agent that fails at step seven and restarts from step one isn't broken. It just wasn't designed for a world where things fail. Build for failure from the start, or spend your incident response time rebuilding it later.