When a language model like ChatGPT gives a wrong answer, the problem might be an unclear question or the model's lack of knowledge. Existing methods for measuring uncertainty cannot distinguish these two causes.
This work introduces a way to tell them apart without needing to see inside the model. The method generates several answers to the original question and also to slightly different phrasings. By comparing how much answers vary within a phrasing versus across phrasings, it separates ambiguity from knowledge gaps.
The approach matches the accuracy of methods that require internal model access. It can guide whether to ask for clarification or supply missing information, making AI systems more reliable.
