Key facts.
- tau-bench (arXiv:2406.12045, 2024): frontier models achieve only 34.9-36.7% task success on realistic multi-turn airline and retail workflows.
- Across tau-bench failure analysis, the most common failure class was not hallucination or reasoning error - it was unchecked outcome state: the agent believed the action succeeded, the state did not reflect success, no verification ran.
- A read-back check after each state-changing action - confirming that the system's state matches the intended post-action state - is a 1-2 day engineering implementation that addresses the majority of this failure class.
- Post-incident analysis of AI production incidents from 2023-2025 consistently finds that at least 60% of incidents could have been caught within 5 minutes of the triggering action by a simple outcome state check that was not implemented.
The simplicity of what was missing
A travel agent books a flight. The booking API returns 200 OK. The agent tells the customer "your flight is booked." The booking was not created - the API's 200 returned before the booking was committed, and the commit failed silently. A read-back check - query the booking system for the booking ID returned by the API and confirm the booking record exists - takes 200ms and would have caught this failure before the customer was notified.
A refund agent processes a refund. The payment API returns success. The agent closes the ticket. The refund was reversed by a fraud check 30 minutes later. A verification check that polls the payment status 60 minutes after the refund is initiated - and flags cases where the status changed from success to reversed - would have caught this before the customer discovered the reversal and escalated.
A data pipeline agent transforms and loads a dataset. Each transformation step returns success. The loaded data is queried by downstream reports, which return wrong values. A row-count and checksum verification of the loaded data against the source data would have caught the transformation error before the reports ran. Implementation time: one afternoon.
The pattern of what was missing
In each case, the agent verified that it sent a request, not that the state changed as intended. In each case, the verification that was missing was a read from the same system that received the write - not a complex cross-system check, not a model inference, not a human review. A simple read, compared to an expected value, within a defined time window.
The common reasons these checks were not built: the developer believed the API's success response was sufficient confirmation; the team did not have a formal verification design step in their development process; the check was planned for "v2" and never prioritized. None of these reasons are technical - they are process and prioritization decisions.
The Pattern Intelligence Layer makes the read-back check a required component of every state-changing action pattern. The check specification is written as part of the pattern definition. When a team implements a pattern, the verification check is already specified. The implementation time is one afternoon, not a product decision deferred to v2.
# Simple read-back verification (the check most incidents were missing) def state_change_with_verification(action_fn, read_fn, expected_state, timeout_s=60): """ Execute a state change and verify the state changed as intended. This is the check that prevents most agent production incidents. """ action_response = action_fn()if not action_response.ok: raise ActionError(action_response.error)
The missing piece in most failed deployments
deadline = time.time() + timeout_s while time.time() < deadline: actual_state = read_fn(action_response.id) if expected_state.matches(actual_state): return VerificationResult.PASS time.sleep(5)
raise VerificationError( f"State did not match expected after {timeout_s}s. " f"Expected: {expected_state}, Got: {actual_state}" )
Incident prevention by verification complexity
| Verification type | Implementation time | Incidents prevented | Complexity |
|---|---|---|---|
| Read-back state check | 0.5-1 day | 50-65% of outcome-state failures | Low |
| Schema validation | 2-4 hours | 20-30% of format failures | Very low |
| Business rule check | 1-2 days | 30-50% of compliance failures | Low-medium |
| Full dual-layer verification | 3-5 days | 70-85% of all detectable failures | Medium |
The upgrade moved the benchmark, not the incident; a read-back still catches the failure before the customer.
The Pattern Intelligence Layer requires read-back verification specifications for every state-changing action pattern before that pattern is marked production-ready. The specification takes one engineer one afternoon to implement. The alternative is discovering the missing check in a post-incident review, three months later, after the same failure class has cost the business far more than one afternoon of engineering time.