Efficiency & systems research

The frontier

Adaptive reference attention reuse speeds up conditioned diffusion transformers

The authors propose RefAdapt-DiT, a training-free framework that adaptively controls joint attention computation between reference images and target generation in diffusion transformers. By monitoring reference drift and target-to-reference attention, it selectively reuses stale reference states, achieving speedups of up to 3.54× on image editing with minimal quality loss.

Jian Tang, Jiawei Fan, Qiannan Zhou +3

Efficiency & systems

Pruning experts by replaceability, not magnitude, preserves reasoning in MoE models

The authors introduce RAZOR, a training-free pruning method for mixture-of-experts models that scores experts by how well surviving experts can compensate for their removal. Across four MoE models (GLM-4.7-Flash, Qwen3.6-35B-A3B, DeepSeek-V4-Flash-0731, Hy3) at 25% and 50% pruning budgets, RAZOR achieved the highest macro average over nine reasoning tasks, outperforming the REAP baseline by up to 5.59 points. However, pruned models still exhibited shifts in response diversity, formatting, and termination.

today

Efficiency & systems

Training-free inference framework trims redundant computation and memory in looped Transformers

FlashLoop is a training-free inference framework that reduces cross-loop redundancy in looped Transformers through token-sparse updates, sparse attention, and KV-residual quantization. The authors show that as loops progress, state changes concentrate on few tokens, attention-output differences concentrate on few key columns, and KV residuals become quantization-friendly. Across several looped models, FlashLoop achieves lossless accuracy with up to 1.64x speedup and 6x KV-cache memory reduction.

today

Efficiency & systems

Mentored decoding can trade small distribution shifts for better predictions

Tran-Thien and Nock formalize mentored decoding, in which a fast draft model’s tokens are accepted while the resulting distribution remains within a chosen distance of a larger target model. Their analysis shows how this controlled deviation can increase draft acceptance and, under boosting-style conditions, produce predictions better than the target’s rather than merely approximate it.

today

Reinforcement learning

Canopy learns when tree-based shortcuts fail in LLM inference

Jerge and Jana present CANOPY, a multi-fidelity bandit algorithm for searching tree-structured candidate spaces without assuming that nearby branches always have similar value. The authors prove that its extra cost scales additively with the number of smoothness violations, and report matched-budget gains across model routing, candidate identification, test-time search, prompt trimming, and prefix caching.

today

Efficiency & systems

Sparse feed-forward activations cut language-model inference costs substantially

Lomeli and colleagues introduce model casting, a mid-training procedure that makes feed-forward network activations sparse enough for dedicated inference kernels to skip most inactive computation. They also propose LoPA Gating, which reduces the cost of identifying active units and surpasses the theoretical threefold speedup limit of conventional gated feed-forward layers.

today

Efficiency & systems

Masked co-adaptive fine-tuning cuts expert fetches by over 23% for MoE models.

Wu et al. propose MaskCoFT, a fine-tuning method that uses learnable binary masks to restrict each layer's Top-K routing to a subset of experts while allowing those experts to adapt to the redirected tokens. For Mixtral-8x7B and DeepSeek-V2-Lite, the method reduces expert fetches per token by 23.7% and 10.1%, respectively, and lowers time per output token by up to 16.4% and 5.5% in a real offloading system. Average accuracy over nine benchmarks remains slightly above the base model.

today

All Efficiency & systems papers →

Every paper read and written up by the research desk from the daily arXiv scan · threads are maintained lines of inquiry with running syntheses