Key facts.
- MIT NANDA research on agentic AI in hospital settings documented pilot-to-production failure patterns where agents that performed well in supervised pilot conditions degraded substantially in autonomous production, with verification layer absence as the primary contributing factor (MIT NANDA, nanda.media.mit.edu).
- Gartner's 2024 analysis found that AI pilot success rates exceed 70% while production deployment success rates at comparable scale fall below 30%, a divergence consistent with pilot conditions masking the failure modes that production conditions expose.
- Five pilot conditions that mask verification gaps: (1) curated task set (no novel variants), (2) complete and clean data (no null fields or format anomalies), (3) supervised sessions (human oversight catches errors before propagation), (4) small scale (errors affect few users before discovery), (5) high-quality feedback loop (errors are rapidly identified and corrected).
- Production removes all five conditions simultaneously: task variety increases, data quality degrades, supervision decreases, scale increases, and the feedback loop slows. Verification gaps that were invisible in the pilot become consequential failures at scale.
- The specific verification gaps most often revealed by the pilot-to-production transition are: novel task type handling (no verified output format for task types not in the pilot set), data quality handling (no validated behavior when required fields are missing), and autonomous error recovery (no tested path for self-correction when verification fails without human oversight).
Why pilots systematically hide verification gaps
A well-run agent pilot is, by design, a best-case scenario. The task set is chosen to showcase capabilities. The data is prepared or curated to be representative in the narrow sense of matching the agent's training distribution. Human monitors are present to catch edge cases. The sample is small enough that even a 20% silent failure rate produces only a few incidents, all of which get fixed before the next demo cycle.
This is not dishonesty on the part of pilot teams. It is the structural consequence of designing pilots to answer "can this work?" rather than "does this work for all the cases production will throw at it?" The pilot answers the first question. The production deployment answers the second. If the second answer is different from the first, the difference is usually found in the cases the pilot did not cover - which are exactly the cases where verification layers are most important.
Stress-testing pilots for verification gaps before production
The stress tests that reveal verification gaps before production deployment are: novel task injection (introduce task types not in the pilot set and observe whether the agent handles them or fails silently), data quality degradation (introduce missing fields, format anomalies, and edge case values and observe whether validation catches them), supervision removal (run a set of sessions with no human monitor and compare outcomes to supervised sessions), and scale stress (run 10x pilot volume over a weekend and analyze the tail of the output distribution). Any of these that reveals silent failures points to a verification gap that needs to be addressed before production.
Pilot conditions vs production conditions by verification gap type
| Pilot condition | Production reality | Verification gap it hides | Pre-production stress test |
|---|---|---|---|
| Curated task set | Novel task variants | Unknown output format handling | Novel task injection |
| Clean data | Missing or anomalous fields | Null/edge case handling | Data quality degradation test |
| Supervised sessions | Autonomous operation | Error recovery without human | Unsupervised session batch |
| Small scale | Production volume | Tail behavior at statistical scale | 10x volume stress test |
| Fast feedback | Slow signal from business outcomes | Slow-propagating silent failures | 48-hour silent failure audit |
MIT NANDA found hospital agents that aced pilots degrade in production, and capability did not explain it; a stronger model meets the same wall. (MIT NANDA)
VibeModel's Pattern Intelligence Layer applies the production failure patterns from your own deployment and the broader agentic AI failure corpus to predict which verification gaps your pilot is hiding. By comparing your pilot's task distribution, data characteristics, and verification coverage against the patterns that predict production failures, it surfaces the specific gaps to close before you scale. That converts pilot success from a reason for confidence to a checklist for production readiness.
Frequently asked questions
How long should a pre-production stress test run before we consider the results representative?
Long enough to accumulate at least 500 task completions per novel condition introduced. For low-volume use cases, two weeks of stress testing may be required to hit this threshold. For high-volume use cases, a weekend may be sufficient. The 500-task threshold provides enough statistical power to detect silent failure rates above 2%.
What is the right metric to track during a pre-production stress test?
Three metrics in combination: explicit error rate (caught by existing validation), deferred audit failure rate (detected in a sample of completed tasks by manual review), and re-engagement rate (tasks where the human user re-opened or followed up, suggesting the agent's output did not fully resolve the need). Together, these capture all three classes of verification gap.
Should we delay production deployment until all verification gaps from stress testing are resolved?
Not necessarily. Prioritize by severity: verification gaps that produce regulatory violations or patient-safety-level impacts must be resolved before production. Gaps that produce silent failures with high remediation cost should be resolved or have human escalation paths added. Gaps that produce low-cost recoverable failures can be addressed in the first production sprint with monitoring in place.