The authors converted eight multiple-choice benchmarks (text, image, video) that group questions by shared contexts such as passages, images, or videos.
Answer pooling revives saturated multiple-choice benchmarks by withdrawing elimination credit
Without writing new items, the label-free conversion lowers guessing floors, quantifies uneven elimination credit, and scores abstention in a single pass.
Academic
Mohamed Eltahir · Abobaker Ahmed · Nawaf Barebood · Hussain Bu Subayt · Tanveer Hussain · Naeemullah Khan
King Abdullah University of Science and Technology · Edge Hill University
Research Digest··2 min read
The authors propose AnswerPool, a method that converts grouped multiple-choice benchmarks into harder tasks by pooling all answer options from questions sharing a context.
Why this paper
From King Abdullah University of Science and Technology and Edge Hill University
In one line
Pooling answer options from related multiple-choice questions creates a harder evaluation task and reveals how much models rely on eliminating wrong answers.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§