Pruning visual tokens is a standard way to cut compute in multimodal LLMs, but the authors show that the common recipe of keeping the highest-ranked native patches is fragile.
Bifocal pruning keeps fine-grained accuracy while using quarter of visual tokens
A training-free two-stage method balances query-focused and image-intrinsic evidence, retaining over 90 percent of base performance on OCR and visual grounding.
Top University
Peiran Yin · Hao Wu · Zihao Yang · Fengyu Zhou · Yijuan Liang · Haozhe Hu · +3 more
Eastern Institute of Technology · Shanghai Jiao Tong University · Uppsala University · University of Science and Technology of China
Research Digest··3 min read
The authors evaluate ten visual token pruning methods across three multimodal LLM families and find that strong average performance masks severe drops on fine-grained tasks.
Why this paper
From Shanghai Jiao Tong University and 3 others
In one line
Bifocal prunes visual tokens in MLLMs, retaining over 90% performance across all ability groups with a quarter of tokens.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§