Law 22 · Instruction & Output
Show, Don't Tell
When prose fails, stop writing prose.
The principle
If an instruction has produced the wrong result twice, writing it a third time more carefully rarely helps, because prose is always open to interpretation. Two or three concrete input and output examples kill the ambiguity that no amount of careful description can. Examples show the rule. Prose only describes it.
Why it happens
Examples pin down what prose leaves open. A written instruction can still be interpreted several ways; two or three input-output pairs show the boundary directly. This is especially useful for edge cases, blanks, rejections, and near-misses. The caveat is that examples are powerful but blunt. Order and formatting can over-anchor the model, so test them the way you test code. If you have rewritten the same instruction twice and the failure remains, stop adding prose. Show the behavior you want.
Watch for
- You have rewritten the same instruction two or three times and the output is still wrong in the same way.
- The model handles the typical case but mangles edge cases the prose tried to describe in the abstract.
- Reviewers keep disagreeing about what the instruction actually means, which means the model cannot resolve it either.
In practice
Your extraction agent keeps formatting phone numbers inconsistently, so you rewrite the instruction a third time: 'normalize to E.164, strip extensions, handle missing area codes gracefully.' It still botches the edge cases. Stop adding adjectives to prose. Drop in four labeled examples instead: '(555) 123-4567' to '+15551234567', 'ext. 12' to dropped, 'unknown' to null, an international number with a country code. The examples pin down exactly what 'gracefully' meant, which no amount of careful description ever could.
Apply it
- Replace failed prose with two or three labeled input-output examples that demonstrate the exact rule.
- Include the hard cases explicitly: edge cases, the empty or null case, and a near-miss that should be rejected.
- Vary or shuffle example order when testing, since order alone can shift results, and keep the examples consistent in format.
The takeaway
When results are inconsistent, switch from describing to demonstrating. Show worked examples, especially the edge cases and the 'leave it blank' cases, and let the model generalize from them.