When you ask an AI system a question, a correct answer might mean it understood. But the information it saw may be incomplete, so a correct answer could be luck. We need to know if it had enough evidence.
The researchers tested this with device logs. They made rules to decide when a visible history determines the answer. They tested two AI models on cases where the history was insufficient and recorded how often they gave correct answers or abstained.
The models often gave answers even when evidence was insufficient. Qwen gave correct answers in 87 such cases and abstained only 7 times. This shows that correctness alone is not enough; we must evaluate whether the AI truly had the needed evidence.