Even strong models misread magnitudes and units. In tests of numerals and units of measurement, models mistake a number's magnitude, reading hundreds of millions as billions, and get the conversion factor between units wrong (NUMCoT, arXiv:2406.02864, 2024). Hand a tool's raw cents, milliseconds, or locale dates straight to the model and it will confidently transform them wrong.
Key facts.
- Models misread magnitudes and unit conversions: in numeral-and-unit reasoning tests, errors concentrate on misidentifying a number's magnitude and the conversion factor between units (NUMCoT, arXiv:2406.02864, 2024).
- Models struggle to produce correct structured transformations outside memorized patterns, faltering on algebraic generalization and context-dependent rules (StructEval, arXiv:2505.20139, 2025).
- The reliable fix is to parse, convert, and compute in deterministic code with typed, unit-tagged outputs, and reserve the model for judgment over already-clean values (data-engineering practice).
Why is the model unreliable at this?
Cents read as dollars and the action is confidently wrong; a stronger model still botches the arithmetic late. (arXiv:2406.02864)
Because next-token prediction prioritizes a coherent answer over a precise one. A tool returns raw data, an amount in cents, a duration in milliseconds, a date in a locale format, a numeric value encoded as a string, and the agent pipes it straight into its reasoning as trusted context. It then has to silently parse, convert, and sometimes compute over it, and that low-level work is exactly where a fluent generator slips: it assumes dollars when the field was cents, treats a string as a number, applies the wrong scale in a derived calculation, or misreads an ambiguous date. The data was correct. The model's unspoken transformation of it was not, and nothing checked the transformation before it became an action.
Why does it stay invisible until it hurts?
Because the wrong value is still a plausible value. An amount off by a factor of a hundred, a duration off by a thousand, a date a month early, all look like normal outputs, so no parser throws and no status code goes red. The error only surfaces when the downstream effect is wrong: a refund of the wrong size, a timeout set in the wrong scale, a report with figures that do not reconcile. This is the verification gap, where the output passes every surface check while being semantically wrong, and it is worst precisely on the messy, ambiguous data where models already misread magnitudes and units.
# Don't let the model infer scale. Normalize in code, tag the unit. raw = call_tool() # {"amount": 4900, "amount_unit": "cents", "dur": 1500, "dur_unit": "ms"} amount = Money(raw["amount"], "USD") / 100 # explicit: cents -> dollars duration_s = raw["dur"] / 1000 # explicit: ms -> seconds # Agent reasons over typed, unit-tagged values, not a raw blob it must interpret.
How do you make the transformation reliable?
Take the low-level work away from the model and give it clean data. Return tool outputs as typed, schema-validated objects that carry explicit units and provenance, so a value is never an ambiguous bare number. Do parsing, unit conversion, normalization, and any derived arithmetic in deterministic code or a code-interpreter step, not in the model's head, and normalize to standards, ISO 8601 for dates, ISO 4217 for currency, explicit units for everything else. Reserve the model for semantic judgment over the cleaned values. Add format and unit variations to your tests, and watch for silent type or scale drift in production traces. The agent then reasons over data that has already been made correct.
Where the silent transformation breaks
| Tool returns | Agent assumes | Fix |
|---|---|---|
| Amount in cents | Dollars | Explicit unit field + code conversion |
| Duration in ms | Seconds | Unit-tagged value, convert in code |
| Number as a string | Already numeric | Typed schema, parse and validate |
| Locale date | Day/month order it expects | Normalize to ISO 8601 |
| Derived calculation | Model does the math | Compute in deterministic code |
The pattern is that tools return ground truth in whatever messy shape the API chose, and the model's confident, silent transformation of it is the weak link, not the data. Normalize and compute in code, hand the agent typed, unit-tagged values, and reserve the model for judgment, and accurate data stays accurate through to the action. Putting a deterministic transform-and-validate layer at the tool boundary, rather than trusting the model to infer scale, is reliability at the pattern level, which is what VibeModel builds as the Pattern Intelligence Layer.
Frequently asked questions
Can't I just prompt the model to be careful with units?
A prompt helps a little but the model still infers scale probabilistically. The reliable fix is to convert and compute in deterministic code and hand the model values that already carry explicit units.
Where should the transformation live?
In the tool wrapper, before the result reaches the model. Return typed, validated, unit-tagged objects so the agent never has to guess whether a number is cents or dollars.
Why is arithmetic a problem for the model?
Because it generates a plausible answer rather than a computed one, so derived calculations can be subtly wrong. Run the math in code and let the model reason over the result.