The authors froze a Sa2VA segmentation model and trained identical lightweight probes to predict masks from either the [SEG] token’s hidden state or its attention over image patches.
Language model attention gives segmenters a clearer spatial target
AttnSeg passes patch-level attention maps directly to a mask decoder, improving localization of objects and object parts.
Big Tech
Eslam Abdelrahman · Hamza Omar Alkaf · Yuheng Li · Siddharth Srinivasan Iyer · Nanxuan Zhao · Eli Shechtman · +2 more
King Abdullah University of Science and Technology (KAUST) · Adobe Research
Research Digest··3 min read
Abdelrahman and colleagues test how a vision-language model should communicate a language-specified region to a segmentation decoder.
Why this paper
From Adobe Research and King Abdullah University of Science and Technology (KAUST)
In one line
Using [SEG] token attention instead of its hidden state as the decoder prompt improves language-guided segmentation accuracy.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (params 4B)
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§