Key facts.
- SWE-bench Verified (2024): top narrow coding agents resolve 40-50% of real GitHub issues end-to-end - the highest reliability tier in deployed agentic systems.
- Narrow agents have bounded action spaces: a coding agent has 8-12 distinct action types. Verification contracts per action type are achievable. A general agent with hundreds of possible actions cannot achieve the same verification coverage.
- The most reliable narrow agents (customer service, coding, data extraction) all implement pre-action input validation, post-action outcome verification, and end-of-task goal achievement check as standard components.
- General agents skip verification because they cannot enumerate what to verify - there are too many possible actions. Narrowness makes comprehensive verification design tractable.
What narrow agents make tractable
A coding agent can enumerate its action types: read file, write file, run test, search codebase, apply patch, submit PR. For each action type, a verification contract can be written: after read file, confirm the file content is non-empty and the file path resolves to the target file. After apply patch, run the test suite and confirm no previously passing tests now fail. After submit PR, confirm the PR is in the target repository and the diff matches the intended changes.
Each of these contracts is written once, tested once, and applied every time that action type executes. The verification overhead per action is low because the contract is pre-built. The coverage is high because the contract covers every execution of that action type, not just selected instances.
A general agent has no bounded action type set. It can call arbitrary APIs, generate arbitrary outputs, and execute arbitrary plans. Writing a per-action-type verification contract for an unbounded action space is not tractable. Teams default to outcome-only verification, which catches failures only after the full action sequence has run and the damage has accumulated.
Applying narrow-agent lessons to general agents
The lesson from narrow agents is not "build narrow agents instead of general ones." It is "identify the bounded action types in your general agent and write verification contracts for each." Most general agents in production actually have a bounded set of high-frequency, high-consequence action types - even if the agent theoretically can do more. Those high-frequency types account for 80-90% of executed actions. Verification contracts for those types cover most of the reliability surface.
Action type enumeration as a design step: before building the agent, list every action type it will take in the target domain. For each type, write the verification contract. Anything without a verification contract is either too rare to prioritize or too ill-defined to automate - and either conclusion is valuable signal during design.
# Narrow agent verification registry pattern
VERIFICATION_CONTRACTS = {
"read_file": lambda result, params: (
result.content is not None and
result.path == params["target_path"]
),
"apply_patch": lambda result, params: (
run_tests().all_passed and
git_diff() == params["expected_diff"]
),
"submit_pr": lambda result, params: (
pr_exists(result.pr_id, params["repo"]) and
pr_diff_matches(result.pr_id, params["patch"])
),
}def verified_action(action_type, action_fn, params):
result = action_fn(params)
contract = VERIFICATION_CONTRACTS.get(action_type)
if contract and not contract(result, params):
raise VerificationError(f"Contract failed for {action_type}")
return result
Verification coverage by agent type
| Agent type | Action types | Achievable contract coverage | Typical reliability tier |
|---|---|---|---|
| Narrow coding agent | 8-12 | 80-100% | 40-50% task completion (SWE-bench) |
| Narrow customer service agent | 10-15 | 70-90% | High (domain-specific) |
| General orchestration agent | 50-200+ | 10-20% | Low in production |
| General agent with domain limiter | 20-40 in-domain | 50-70% | Medium with right constraints |
Shipping agents win on per-action checks, not an upgrade; the gain survives a frontier model and saves the rework.
The Pattern Intelligence Layer enforces action-type enumeration as a pre-design step for every pattern. A pattern that does not have enumerated action types with verification contracts cannot be published as production-ready. The narrow-agent lesson - that reliability follows verification coverage, which follows action-type boundedness - is encoded into the pattern specification process itself.