Skip to content
Search
AppliedGlossary

Hallucinated Citation

Models write citations the same way they write everything else, by predicting what plausibly comes next, so a made-up case name with a realistic docket number is a natural failure (hallucination). In Mata v. Avianca, a federal court in New York sanctioned two lawyers and their firm in June 2023, including a $5,000 penalty, after they filed a brief that cited cases that didn't exist.

Retrieval reduces the problem without removing it. In a 2024 preregistered study of commercial AI legal research tools built on retrieval, each tool still hallucinated between 17% and 33% of the time. The fix is procedural. Have the model quote from documents you supplied (grounding and RAG), require a source for every citation, and have a person or a script confirm that each source exists and supports the claim. Chain-of-verification helps the model catch some of its own errors first. Qurtoo's guide to legal document review prompting applies this to contracts.

Example

Answer using only the documents inside <sources>.
For every claim, cite the document ID and quote the exact sentence that supports it.
If the sources don't support a claim, write "not in sources" instead of citing anything.
Don't cite cases, papers or web pages that aren't in <sources>.

<sources>
[doc-1] ...
[doc-2] ...
</sources>

Question: {question}

When to use it

  • Legal, medical, academic or financial writing where sources matter
  • Any AI draft that includes links, case names, statistics or paper titles
  • Reviewing work a colleague produced with an AI tool

When not to use it

  • Assuming a citation is real because it's formatted correctly
  • Trusting that retrieval alone makes citations safe