Law 12 · Retrieval & Memory
Grounding Is Not a Guarantee
Retrieval reduces hallucination. It doesn't eliminate it.
The principle
Vendors marketed RAG legal tools as 'hallucination-free', but a Stanford audit found they still made things up 17 to 33% of the time. Handing the model a source doesn't force it to use that source faithfully. It can misread it, over-generalize, or cite a real document for a claim the document never makes. Grounding lowers the error rate. It never gets it to zero.
Why it happens
A source in the prompt nudges generation; it does not bind it. The model can cite a real document while making a claim the document never supports, over-reading a narrow passage, or combining two spans into an unsupported synthesis. That is why grounding benchmarks check whether each claim is entailed by the provided text, not merely whether a citation exists. Retrieval lowers hallucination risk, but it does not make the system hallucination-proof. The verification unit has to be the claim, tied to the exact span that supports it.
Watch for
- A grounded system is described to stakeholders as hallucination-free or hallucination-proof.
- No step checks that each generated claim is actually entailed by a retrieved span.
- Citations are attached to answers but nobody has verified the cited passage supports the specific claim.
In practice
Your team ships a contracts assistant, tells the client it is 'hallucination-free because it uses RAG', and a month later it cites a real clause for an indemnity term that clause never mentions. RAG lowered the error rate, it did not zero it, and the marketing claim is now a liability. Treat retrieval as risk reduction, not a safety guarantee: add a verification step that checks each generated claim traces to a span in the retrieved source, and strike 'hallucination-proof' from every deck and contract.
Apply it
- Add a verification pass that checks each output claim is entailed by a specific retrieved span before returning it.
- Require inline attribution at the claim level so faithfulness can be audited rather than trusted.
- Frame retrieval as risk reduction in all messaging and remove absolute safety language from decks and contracts.
The takeaway
Treat 'we use RAG' as risk reduction, not a safety guarantee. Check that generated claims actually trace back to the retrieved passage, and never sell a grounded system as hallucination-proof.