AI Vocabulary
Hallucination: why is the false citation always the most convincing?
A hallucination is an assertion produced by an AI system with the same assurance as an accurate one, corresponding to nothing. In legal work it usually takes the form of a plausible, non-existent citation: correct court, correct format, a holding consistent with the argument, and no decision behind it.
Why it happens
A language model does not consult a database when it answers. It produces the most probable continuation of a text, from what it has learned. A case citation has a highly regular form — court, division, date, number — and that form is perfectly imitable without the content existing.
The system is not lying and is not mistaken in the way a person would be. It produces something that looks like a citation, because that is what was asked of it.
What it is not
Not a reasoning failure. A system can hallucinate a source while reasoning soundly, and that is what makes it dangerous: the text reads well.
Not a defect a more powerful model eliminates. Recent models hallucinate less, they still hallucinate, and their hallucinations are harder to spot precisely because the rest of the text is better.
And not particular to one provider. The French decisions of late 2025 that identified invented case law in court filings do not distinguish between the tools used.
An associate errs where they hesitate, and the hesitation is visible. A system produces an error with the same assurance as an accurate statement.
An example, to fix ideas
You ask whether there is authority on a precise point. The system answers by citing a Court of Appeal judgment, with its date and number, summarising the holding in two lines perfectly suited to your question.
Everything is plausible: the court exists, the date is coherent, the number has the right format, and the holding is exactly what one would expect. There is nothing to spot, short of opening the decision. That is why such citations pass review by competent lawyers.
How it is contained
Three measures, in order of effectiveness.
Supply the sources rather than have them produced. A system that searches a real corpus before answering cannot invent a decision it has not found — it can, however, misread one, which is a different problem.
Require every citation to be openable in one action. A citation that cannot be opened will not be opened, and an unopened citation is an unverified one.
Set an absolute rule rather than a proportionate regime: no cited source appears in outgoing work without someone having opened it.
What it does not solve
No architecture eliminates the phenomenon. Frequency falls, verification becomes practicable, no guarantee is obtained. A provider claiming otherwise would be describing its own system inaccurately.
And verification has a cost that does not disappear. Opening 40 citations takes time; what can change is that the time be seconds per citation rather than minutes.
Why it matters to a lawyer
Because risk is not measured by the number of errors but by those that get through. An error rate that falls while the detectability of the remainder collapses is an unfavourable arrangement for someone whose signature engages them.
And because the criticism, when it comes, does not bear on the use of a tool. It bears on the failure to verify, which is an old obligation.