Agents report a tool call as a success while the business outcome quietly fails.Effects externalize immediately and there's no principled signal that no earlier work remains, so an agent confirms "done" on an async action the queue, saga, or downstream service never actually completed (Atomix: Timely, Transactional Tool Use for Reliable Agentic Workflows, arXiv:2602.14849, 2025). The order is confirmed; it never shipped.
Key facts.
- Agents report tool success while the business outcome fails: effects externalize immediately with no principled signal that no earlier work remains, so a confirmed action can be incomplete downstream (Atomix, arXiv:2602.14849, 2025).
- Acknowledgment is not outcome. A 200 or a function return means accepted. The write, the settlement, the fulfillment, those complete asynchronously, and can still fail downstream (distributed-systems eventual-consistency model).
- Idempotency keys and saga-style compensations are standard safeguards. Both assume a single linear workflow and break under parallel or multi-agent execution. A real completion signal is still required (Atomix, arXiv:2602.14849, 2025).
Why does a 200 not mean the work happened?
Most real systems are async behind a synchronous-looking call. The agent emits the function call, gets a result back, feels like it completed. It didn't. The backend accepted the request. It enqueued a message, started a workflow, fired a webhook, wrote a record for a later step to validate or compensate. The immediate 200 means accepted for processing. The queue consumer can crash, a saga can trigger a compensating transaction, an eventual-consistency conflict can resolve the other way, or an external system can reject the item, all after the agent already moved on. The agent treated an acknowledgment as ground truth, so its report of success describes a step that may never have finished.
Why is this failure so hard to see?
Because every signal the agent has says success. The tool returned cleanly, no exception fired, and the trace shows the call completed, so benchmarks that score on transport-level success would call it a win. Only an evaluation that checks the actual end state reveals the gap, because a clean tool response doesn't mean the task completed. The divergence lives in a system the agent never observes, so it surfaces downstream: the order marked confirmed that never shipped, the payment shown captured that the settlement later reversed. The agent isn't lying. It reported the only thing it could see, the acknowledgment, and called it the outcome.
# Make tools return the outcome, not just an ack; verify the real state result = submit_order(cart, idempotency_key=key) # result: {"accepted": true, "order_id": "...", "business_status": "PENDING"} final = await confirm_state(result.order_id) # poll / callback for the real outcome assert final.business_status == "SHIPPED" # not "accepted", not "200": shipped
How do you make success mean the outcome?
Stop treating the acknowledgment as the result. Model every side-effecting tool call as the start of a workflow: attach a correlation ID, and get the final state through a completion callback, a poll, or an outbox, rather than trusting the first 200. Have tools return structured results that separate acceptance from completion, with fields like accepted, business_status, and requires_follow_up, and add deterministic checks the agent must pass before it reports done. Instrument business-outcome signals, order fulfilled, inventory reserved and consistent, payment settled, not just HTTP status. Use idempotency keys so a follow-up or retry can't duplicate the action. The agent then confirms the thing happened, instead of confirming that someone agreed to try.
Acknowledged versus actually done
| The agent saw | What it meant | What to verify instead |
|---|---|---|
| 200 from submit | Accepted for processing | Final business_status via callback/poll |
| Function returned | Request enqueued | Queue consumer completed the work |
| Payment 200 | Authorized, maybe not settled | Settlement confirmed downstream |
| Record write OK | Write accepted, not reconciled | Consistency after compensation |
| No exception | Nothing crashed | The real-world effect occurred |
The pattern is that an agent only sees the acknowledgment, and in an asynchronous system the acknowledgment isn't the outcome, so success can be reported for work that never finished. Model the call as a workflow, return the business status, and verify the final state with a correlation ID and idempotency, and the agent confirms reality instead of intent. Checking the downstream outcome rather than the acceptance is reliability at the pattern level, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
Will a smarter agent notice the order never actually shipped?
A 200 means accepted, not shipped; the queue fails unseen, and a more capable agent reports the same success. (arXiv:2602.14849)
Isn't a 200 supposed to mean success?
It means the request was accepted, which in an async system isn't the same as completed. The work happens later in a queue or workflow that can still fail, so verify the final business state, not the acknowledgment.
How does the agent learn the real outcome?
Give it a completion signal: a callback, a poll on a correlation ID, or an outbox the runtime reconciles. The tool result should carry a business_status the agent checks before reporting done.
Why do idempotency keys matter here?
Because following up or retrying to confirm an outcome must not trigger the action a second time. An idempotency key lets you re-check safely without duplicating the order or the charge.