News without the noise
Baser et al. test whether knowledge edits and unlearning remain effective when an adversary changes how an input is divided into tokens, while preserving the underlying text. Across five language models, six datasets and six modification techniques, alternative tokenizations frequently restored the model’s pre-modification response.
Zhang et al. propose a framework for multi-task models that learns explicit cross-task relationships by approximating the joint distribution of task labels via pairwise interactions. Tested on YouTube's Notifications, Homepage, and Watch Next surfaces, the method improved both accuracy and user satisfaction metrics without the complexity of full joint modeling.
todayWang et al. introduce OracleLadder, a diagnostic tool that provides increasing levels of oracle help to large language models on multi-step math problems. By giving models a roadmap of sub-goals and their answers, they isolate failures into five categories. The dominant failure mode across all tested models is a composition gap: models can solve each intermediate step alone but cannot compose them into the correct final answer.
todayYu and colleagues test whether an on-policy context-distillation teacher should receive each problem’s gold answer or a short instruction aimed at common student errors. Across three logic datasets and two language-model families, matched instructions retained in-domain performance while improving out-of-distribution accuracy by 4 to 17 percentage points in seven of eight comparisons.
todayThe authors introduce RAZOR, a training-free pruning method for mixture-of-experts models that scores experts by how well surviving experts can compensate for their removal. Across four MoE models (GLM-4.7-Flash, Qwen3.6-35B-A3B, DeepSeek-V4-Flash-0731, Hy3) at 25% and 50% pruning budgets, RAZOR achieved the highest macro average over nine reasoning tasks, outperforming the REAP baseline by up to 5.59 points. However, pruned models still exhibited shifts in response diversity, formatting, and termination.
todayFlashLoop is a training-free inference framework that reduces cross-loop redundancy in looped Transformers through token-sparse updates, sparse attention, and KV-residual quantization. The authors show that as loops progress, state changes concentrate on few tokens, attention-output differences concentrate on few key columns, and KV residuals become quantization-friendly. Across several looped models, FlashLoop achieves lossless accuracy with up to 1.64x speedup and 6x KV-cache memory reduction.
todayGahtan et al. introduce effective depth, a scalar that treats a transformer's residual stream as a discrete-time process and measures how representation similarity decays with layer distance. Across 16 decoder-only LLMs, they find that effective depth is almost always below a closed-form reference that assumes maximally diverse orthogonal updates, suggesting that residual updates are strongly correlated.
todayEvery paper read and written up by the research desk from the daily arXiv scan · threads are maintained lines of inquiry with running syntheses