Vision research

The frontier

Adaptive reference attention reuse speeds up conditioned diffusion transformers

The authors propose RefAdapt-DiT, a training-free framework that adaptively controls joint attention computation between reference images and target generation in diffusion transformers. By monitoring reference drift and target-to-reference attention, it selectively reuses stale reference states, achieving speedups of up to 3.54× on image editing with minimal quality loss.

Jian Tang, Jiawei Fan, Qiannan Zhou +3

Vision

Training-free visual memory framework compresses video tokens while preserving event coherence

The authors propose KeyRec, a training-free method that builds bounded visual memory by maintaining a dense recent cache and a structured event bank. It achieves the best compressed performance in 13 of 15 settings across four benchmarks using only 10% of the decoder-facing visual-token budget, outperforming compressed baselines by 2.21 to 18.37 points on real-time questions.

today

Vision

Decoupled temporal axis yields motion-prioritized video representations efficiently

The authors conduct a controlled architecture-objective study and propose TT-VidT, which uses a DINOv3-initialized ViT-B/16 per-frame spatial path and a compact Temporal Transfer Layer, trained with Diff Compression to reconstruct frames from a first-frame anchor and motion tokens. They show TT-VidT leads fine-tuning on Jester, Something-Something V2, ARID, and Diving48, improving 54–121% over non-TT methods, with 48–55% fewer encoder FLOPs than DisMo and VideoMAE/V-JEPA2.

today

Generative media

Fine-grained routing makes sparse video attention faster without quality loss

Zhang et al. introduce VSA2, a trainable sparse-attention system for video diffusion transformers that selects a small, query-dependent subset of token interactions. The authors report substantially faster 720p generation than a FlashAttention-3 baseline, while maintaining comparable or better video quality.

today

Vision

Pruning visual tokens and decoder sublayers cuts multimodal model computation

Chen and colleagues target two sources of multimodal model overhead: redundant visual tokens and unnecessary computation inside the language-model decoder. Their training-free SPIDER framework retains complementary visual information from middle and deep vision layers, then selectively skips attention or feed-forward sublayers. On LLaVA-NeXT-7B, the authors report 79 percent fewer FLOPs while retaining 96 percent of baseline performance.

today

Robotics & embodied AI

Compressed world models make zero-shot driving plans far faster

Zhu and colleagues evaluate action-conditioned joint-embedding predictive architectures, or JEPAs, as driving world models without training a driving policy. Their proposed AD-E2E-JEPA compresses the latent patches used for planning and reports a 100-fold inference speedup, while retaining planning accuracy and improving representations transferred to imitation learning.

today

Vision

Self-distillation helps vision-language models reason with far fewer visual tokens

Jeddi et al. investigate why reasoning vision-language models falter when most visual tokens are removed. They find that pruning does not always erase the necessary evidence: repeated sampling from the same sparse representation often produces correct answers that greedy decoding misses. Their self-distillation method improves how reliably models use this surviving information, reaching 92.43% of unpruned performance with only 10% of visual tokens retained.

today

All Vision papers →

Every paper read and written up by the research desk from the daily arXiv scan · threads are maintained lines of inquiry with running syntheses