Key facts.
- tau2-bench (arXiv:2506.07982) documents value interpretation errors as a source of agent task failure separate from reasoning errors, occurring when agents receive correctly formatted outputs with ambiguous or context-dependent units or types (tau2-bench, arXiv:2506.07982).
- HammerBench (arXiv:2412.16516) demonstrated that LLMs make assumptions about argument format and type when specifications are ambiguous, applying these assumptions to output parsing as well as input construction.
- Currency handling is a documented source of agent reasoning errors: tool responses that return amounts without explicit currency symbols, or that use localized number formats (1.000,00 vs. 1,000.00), cause downstream calculation errors when agents assume a specific format.
- Timestamp handling across time zones, without explicit UTC offset in the tool response, causes scheduling agents to book appointments in the wrong time zone - a class of failure that is correct at the data layer and wrong at the interpretation layer.
- Floating-point precision in financial tool responses causes downstream rounding differences when agents reason over values without knowing the precision contract - producing results that differ from the authoritative system by fractions of a cent that accumulate at scale.
- tau2-bench traced errors to value interpretation, not reasoning; a better model still misreads units, the gap sits below it. (arXiv:2506.07982)
The interpretation gap between tool output and agent reasoning
A tool response value carries metadata that the response format does not always make explicit: the units of the quantity, the precision contract, the encoding (UTF-8, Windows-1252, ISO-8859), the date format, the currency convention. When this metadata is absent, the agent must infer it from context. Models are trained to make those inferences, and they make them confidently. Sometimes they infer correctly. When they do not, the reasoning proceeds from an incorrect value and the error is attributed to "model hallucination" - when the actual source was the missing metadata in the tool response.
The Mars Climate Orbiter failure is the canonical engineering example: the spacecraft was lost because one system reported force in pound-force per second and another expected newton-seconds, and no validation layer caught the mismatch. Agent tool response interpretation fails for exactly the same reason, at much lower cost per incident, but with much higher frequency.
What output validation at the tool wrapper layer requires
Output validation for tool responses requires a schema that specifies not just field names and data types but also units, precision, encoding, and range. For currency fields: the ISO 4217 currency code must accompany the amount. For timestamps: the UTC offset or timezone designation must be present. For numeric quantities: the unit and precision must be in the schema. For text fields: the encoding must be declared. This is not exotic engineering. It is the kind of schema discipline that database teams apply to every column definition. It has not been consistently applied to tool response schemas for agent consumption.
Data validation requirements by field type
| Field type | Common interpretation failure | Required schema element | Validation check |
|---|---|---|---|
| Currency | Amount without currency code | ISO 4217 currency code | Present and valid currency code |
| Timestamp | No timezone or UTC offset | ISO 8601 with offset | Offset present and parseable |
| Quantity | Missing unit label | Unit annotation in schema | Unit matches expected for field |
| Numeric | Localized decimal separator | Explicit precision and format | Format matches schema declaration |
| Text | Encoding mismatch | Content-Type charset declaration | Encoding declared and matches |
VibeModel's Pattern Intelligence Layer identifies which tool response fields in your specific integrations have historically produced interpretation errors. When a currency field without an ISO code precedes a financial reasoning error in the pattern data, the layer flags that field's validation gap - giving you a specific engineering task rather than a general "validate your outputs" recommendation. The pattern layer turns data quality improvement from a principle into a prioritized task list.
Frequently asked questions
Should we normalize tool responses to a canonical format before passing them to the agent?
Yes, in most cases. Normalizing timestamps to ISO 8601 UTC, currency amounts to value-plus-code pairs, and quantities to value-plus-unit tuples in the tool wrapper layer eliminates a class of interpretation errors before they reach the model. The cost is the normalization logic; the benefit is predictable model input.
What do we do when the tool API does not expose unit or currency information in its response schema?
Document the assumption explicitly in the tool wrapper and treat it as a schema debt item. The wrapper adds the inferred metadata (based on documented API behavior) and flags it as inferred rather than explicit. Any subsequent API change that invalidates the inference will surface in the tool integration tests.
How does this relate to the "garbage in, garbage out" principle for LLM agents?
It is exactly that principle at the tool boundary. Tool responses are inputs to the model's reasoning. If those inputs carry ambiguous or missing metadata, the model reasons over ambiguous data. The output of that reasoning reflects the input ambiguity, not the model's capability limit.