Law 24 · Instruction & Output
Surface Ambiguity, Don't Resolve It
When the data is unclear, don't guess confidently.
The principle
Faced with two plausible matches, conflicting sources, or a missing field, an agent's instinct is to pick the most likely option and move on, a confident choice that quietly buries the doubt. When the stakes touch identity, money, or anything you can't undo, a quiet wrong guess is far worse than an honest 'this is unclear'.
Why it happens
Models are biased toward producing an answer. When two matches look plausible or a field is missing, the easy path is to pick one and move on. That hides the uncertainty from every downstream system. Abstention benchmarks show that even strong models often fail to say a question is unanswerable unless the output format allows it. The fix is structural: unclear must be a valid result, not a failure to comply. Give the agent an explicit abstain, unknown, or escalate path and reward it for using that path when evidence is weak.
Watch for
- The agent commits to one of several plausible matches without recording that alternatives existed.
- A required field is always filled, even when the source data plainly lacks the value.
- Conflicting sources get silently reconciled into a single clean answer with no trace of the disagreement.
In practice
An invoice-matching agent finds two vendors named 'Acme LLC' with different tax IDs and confidently picks the one with the higher historical volume, routing a $40k payment to the wrong account. Nobody notices until reconciliation, because the output looked clean and decisive. The agent should have stopped and flagged it: preserve both candidate records with their tax IDs and source rows, and request a second identifier or a human decision. When money, identity, or anything irreversible is on the line, an honest 'this is ambiguous' beats a tidy wrong answer every time.
Apply it
- Give the agent an explicit way to abstain or escalate, and make unclear a valid, low-friction output.
- On a tie or a conflict, preserve every candidate with its source instead of collapsing to one.
- For irreversible or identity, money, or safety-critical decisions, route ambiguity to a human or request a second identifier before acting.
The takeaway
Make the agent escalate ambiguity instead of papering over it: ask for another identifier, keep both conflicting values with their sources, or flag the conflict for a human. Push the doubt to whoever can actually resolve it.