Skip to content
Search

Chain-of-Verification Reduces Hallucination in Large Language Models

Authors: Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston

Published September 20, 2023

Read the paper: arXiv 2309.11495 · PDF

Why it matters

Hallucination, a plausible but wrong fact, is still an unsolved problem, and Chain-of-Verification is one of the few fixes you can apply with prompts alone. The key detail is step three: each verification question is answered on its own, so the answer isn't steered by the draft or by the other answers. The authors also tested a joint version that answers the questions in the same prompt as the draft, and found it can repeat the draft's hallucinations. The factored version, with each question answered separately, did better.

The paper shows the effect on three kinds of task, from short lists of facts to longform writing, which makes it a good fit for factual drafts, summaries and research notes.

How to use it

Run it as a short prompt chain. Prompt one: write the draft. Prompt two: list the factual claims in the draft as questions. Prompt three: answer each question in a fresh call, without showing the draft. Prompt four: rewrite the draft, keeping only what the answers support. It costs several extra calls, so save it for content where a wrong fact is expensive. Where you have trusted sources, answer the questions from them instead of from the model's memory, which is grounding.

Abstract

Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models. We study the ability of language models to deliberate on the responses they give in order to correct their mistakes. We develop the Chain-of-Verification (CoVe) method whereby the model first (i) drafts an initial response; then (ii) plans verification questions to fact-check its draft; (iii) answers those questions independently so the answers are not biased by other responses; and (iv) generates its final verified response. In experiments, we show CoVe decreases hallucinations across a variety of tasks, from list-based questions from Wikidata, closed book MultiSpanQA and longform text generation.