The authors distinguish aleatoric uncertainty, which comes from ambiguous or incomplete data, from epistemic uncertainty, which reflects limitations in the model.
Masking irrelevant image regions exposes hidden uncertainty in multimodal models
Causal-Invariant Masking detects hallucinations by measuring how model predictions change when an image is restricted to query-relevant regions.
Big Tech
Haoyang Luo · Linwei Tao · Jie Gui · Xinghao Chen · Chang Xu · Jianyuan Guo · +1 more
City University of Hong Kong · Apple · Southeast University · Huawei Noah’s Ark Lab · University of Sydney
Research Digest··3 min read
Thread:VLM Hallucination Circuits
Luo et al.
Why this paper
From Apple and 4 others · Part of VLM Hallucination Circuits, now 6 papers
In one line
Causal-Invariant Masking isolates epistemic uncertainty via semantic shift under causally invariant transforms, achieving state-of-the-art hallucination detection.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§