Key facts.
- Work on the inherent limitations of autoregressive models shows model-level fallibility is fundamental, so reliability cannot be a model property alone. source
- AssistantBench finds even strong models complete only a small fraction of realistic web tasks, so system-level engineering is what closes the gap. source
- AssistantBench has agents finishing a slice of real work and the autoregressive limits Embers names run deep, so a more capable model is no unit. (arXiv:2309.13638)
Why is the model the wrong unit of reliability?
The instinct is to treat reliability as a model spec: pick a better model and the agent gets more reliable. It helps at the margin and never gets you there, because the model has fundamental limitations that a benchmark score cannot wish away and because the failures that hurt in production, an unverified action, an unrecovered error, an unbounded loop, are not things the model fixes by being smarter. Embers of Autoregression is one articulation of why model-level fallibility is structural and the AssistantBench numbers show what that means in practice: even strong models post low success on realistic tasks. If reliability lived in the model, those numbers would not exist.
Reliability lives in the system. It comes from verifying the model's output before acting on it, from recovering when a step fails, from limits and circuit breakers that contain a bad run, from observability that catches a problem early and from routing the consequential decisions to a human. These are system components, not model parameters and they are what turn a fallible model into a dependable agent, the same way reliable web services are built from unreliable machines. The teams whose agents work did not find a perfect model; they built a reliable system around an imperfect one.
Where does system reliability come from?
| Source | Model-only view | System view |
|---|---|---|
| Correctness | Hope the model is right | Verify before acting |
| Failure | Model retries blindly | Recovery and limits |
| Oversight | None | Observability and human gates |
| Result | As reliable as the model | More reliable than the model |
Building reliability at the system level requires knowing where the model is and is not dependable, which is what VibeModel provides as the Pattern Intelligence Layer. By making the model's reliability explicit at the pattern level, it lets you place verification, recovery and control exactly where the model is weak, so the system as a whole behaves more reliably than the model ever could on its own.
Frequently asked questions
Won't a better model eventually be enough?
No. Model limitations are fundamental and the production failures that matter are system issues like unverified actions and unrecovered errors, which a smarter model does not fix.
What are the system components?
Verification, recovery, limits and circuit breakers, observability and human gates, built around the model rather than inside it.
Is this like classic systems engineering?
Yes. Reliable services are built from unreliable machines via system design and reliable agents are built from fallible models the same way.