Large language models now write text, answer questions, and run tasks, but also invent confident-sounding falsehoods. No one can check everything manually, so can they catch their own mistakes?
The authors built HaluCheck, a large bilingual of twisted facts and unfaithful agent action logs, the step-by-step records of what automated assistants did. Many LLMs tried to detect, locate, and correct hallucinations, that is, fabricated statements, with or without reasoning, web search, or extra training.
Models could often tell that something was wrong, but struggled to pinpoint where the error started, fix it, or detect unfaithful steps in agent logs. The extra aids helped only a little. Current models should not be their own safety check.