Every agent step that has a side effect needs to be safe to run twice. This is not a nice-to-have. It is the difference between a production-grade agent and a demo that breaks on retry.
Agents retry. Frameworks retry on timeout. Orchestrators restart failed sessions. Checkpoint recovery re-executes steps that may have partially completed. The environment your agent runs in assumes that any step might run more than once. If your steps don't account for that, you have a latent data corruption bug waiting for the first production incident to surface it.
What idempotency means for agent steps
An idempotent operation produces the same result whether it runs once or ten times. The side effects are the same. The final state is the same. Running it again after it already succeeded doesn't change anything.
For agent steps, this means: if step N is re-executed because of a retry, crash recovery, or orchestrator restart, the world should end up in the same state as if step N had run exactly once.
This is straightforward for read operations. Reads are naturally idempotent. It's the writes that need work.
The four patterns
Pattern 1: Idempotency keys
Before making an external API call, generate a deterministic idempotency key for that call. The key should be derived from the task ID and step number, not from a random value or timestamp.
idempotency_key = sha256(task_id + ":" + step_number + ":" + step_name)
Pass this key as a header or parameter when making the call. The receiving API uses it to detect duplicate requests: if it's seen this key before, it returns the cached response instead of executing the operation again.
Many payment APIs (Stripe, Adyen), messaging APIs (Twilio, SendGrid), and modern REST APIs support idempotency keys natively. For APIs that don't, implement the deduplication layer yourself: store completed (task_id, step) pairs in a durable store and check before executing.
Pattern 2: Check-then-act
Before executing a write, check whether it's already been done.
def create_user_account(user_id: str, email: str) -> Account:
# Check if already exists
existing = db.query("SELECT * FROM accounts WHERE user_id = ?", user_id)
if existing:
return existing # Already done, return the result
# Safe to create
return db.insert("accounts", {"user_id": user_id, "email": email})
This pattern works for database writes and for API calls that have a discoverable state. The check needs to be in the same transaction as the write (or use an optimistic lock) to avoid race conditions.
Pattern 3: Upsert instead of insert
Replace insert operations with upserts. An upsert creates the record if it doesn't exist and updates it if it does. The result is the same record in the database regardless of how many times the operation runs.
INSERT INTO step_results (task_id, step_name, result, completed_at)
VALUES (?, ?, ?, NOW())
ON CONFLICT (task_id, step_name)
DO UPDATE SET result = EXCLUDED.result, completed_at = EXCLUDED.completed_at;
This is the cleanest pattern for recording step outputs. It's safe under concurrent execution, handles retries naturally, and requires no application-level deduplication logic.
Pattern 4: Saga with compensating transactions
For multi-step operations where partial completion needs rollback, implement the saga pattern. Each step has a compensating transaction that undoes its effect. If a later step fails, the orchestrator applies compensating transactions in reverse order.
This is more complex but necessary for sequences where steps have dependencies that simple idempotency can't handle — for example, booking a flight and a hotel where the hotel is booked only if the flight succeeds.
What to think about for each step type
Database writes: Use upserts with a unique constraint on (task_id, step_name) or (entity_id, operation_type). Never use auto-increment IDs as the idempotency anchor — derive the ID deterministically from the input.
Email and notifications: Pass idempotency keys. Rate-limit by (recipient, template, time window) at the application level as a second layer. Store sent notification records and check before sending.
External API calls with monetary effects: Stripe, payment processors, booking systems. Always use their native idempotency key support. Store the idempotency key and response together. On retry, check whether you already have a stored response for this (task_id, step_name) — if so, use it without calling the API again.
File and object writes: Write to a deterministic path derived from the task ID and step. Overwriting the same file with the same content is idempotent. If your file creation step generates content dynamically (e.g., timestamps or UUIDs in the filename), seed the randomness from the task ID so the same task produces the same filename.
Queue and event publishing: Use a deduplication key on the message. Most message queues (SQS, Kafka, Pub/Sub) support this. The queue deduplicates on the key within a time window — if you publish the same key twice within that window, only one message is delivered.
Testing idempotency before you ship
Your test suite should include a class of tests that run each step twice and verify the world state is identical after the second execution. If you can't write this test, the step is not idempotent.
def test_create_account_idempotent():
# Run once
result_1 = create_user_account("user_123", "user@example.com")
state_after_1 = db.count("SELECT COUNT(*) FROM accounts WHERE user_id = 'user_123'")
# Run again
result_2 = create_user_account("user_123", "user@example.com")
state_after_2 = db.count("SELECT COUNT(*) FROM accounts WHERE user_id = 'user_123'")
assert result_1 == result_2 # Same result
assert state_after_1 == state_after_2 # Same world state (1, not 2)
Agents that checkpoint and recover are safe when every step passes this test. Agents that don't are ticking clocks.
The architecture decision that makes idempotency possible is made before the first line of code: choosing to derive identifiers deterministically from task state rather than generating them randomly at runtime. Make that decision now, not after the first production incident.