News without the noise
New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.
2566 articles · page 37 of 54 · The Research front page
Tan and colleagues translate self-distillation from a training procedure into an inference-time decoding method. Their approach scores candidate answers by how much more likely the frozen model considers them under an excellent-reasoning context than under a poor-reasoning context, then approximates the resulting globally reweighted distribution with beam search.
Aimuyo, Gandhi and Kozyrakis present Purlin, a framework for collective communication among GPUs connected within a high-bandwidth scale-up domain. By separating collective semantics, coordination and hardware data movement, Purlin improves microbenchmark performance and produces smaller but measurable gains in language-model serving and diffusion image generation.
Zhang and colleagues introduce VidScribe, a benchmark of 803 human-verified samples designed to expose failures in generating, preserving and editing text inside video. Across 11 commercial and open-source systems, image-to-video models preserved existing text most reliably, while localized video editing remained the clearest bottleneck.
The authors assembled FineART, a bimanual manipulation dataset containing 40,543 episodes across 151 tasks, with language annotations marking more than half a million intermediate steps. They used it to mid-train FineART-VLA, a vision-language-action policy that selects a next subtask before generating robot actions, and report substantial gains on spatial, long-horizon and cross-robot evaluations.
Finkelshtein et al. introduce the Universal Classifier, a graph model designed to work across datasets with different feature dimensions, label spaces and prediction targets. It converts node, edge and graph tasks into comparisons between queries and labeled anchors in a shared latent space, enabling one pretrained model to perform zero-shot inference on unseen graphs.
Zhang et al. introduce DSpine, a speculative drafter that proposes multiple tokens in parallel while allowing each position to influence its successor across successive network layers. On Qwen3-4B and Qwen3-8B, the method produced longer accepted prefixes than the compared parallel drafters, with a 27.8% gain over DFlash in one principal setting.
Zhang et al. tested whether effective post-training requires examples that humans can read. Their method, Desired-Update-Aligned Synthetic Data (DASA), converts source data into continuous, model-conditioned embeddings that produced comparable or better downstream results than the original text under matched low-rank adaptation settings.
Liu et al. introduce LoopCD, a training-free decoding method for looped Transformers, which repeatedly apply the same parameter block to refine each token representation. By treating an early recurrent state as a weak predictor and contrasting it with the final state, the method improves benchmark performance and can recover full-depth accuracy with substantially less computation.
Ahmad, Seth and Sankarapu reinterpret apparent self-repair after model ablations as the ordinary action of counterweights already embedded in a network. By varying intervention strength rather than testing only deletion, they find that many downstream components respond linearly to the amount of contrastive signal removed or introduced.
Dayan, Eitan and Maron introduce CrossGMN, a metanetwork that jointly represents a trained source model and a differently shaped target model as a connected graph. In compression experiments spanning implicit neural representations and image classifiers, its predicted student weights reduced the subsequent knowledge-distillation time by as much as 8.89 times.
Williamson and colleagues test whether a model’s stated confidence reflects the probabilities encoded in its next-token distribution. Across synthetic transformers and pretrained language models, they find that verbalized and internal probabilities respond to the same evidence and remain aligned beyond what would be expected from merely tracking common inputs.
Liu and colleagues introduce CoVisco, a unified image and video encoder designed to avoid both dense attention across every frame and post-hoc token compression. In a 64-frame configuration, the model represents four temporal segments with 400 abstract tokens and reports video-understanding performance close to, or on some benchmarks better than, OneVision-Encoder.
Zuin and colleagues separate self-distillation with privileged context into three design choices that prior studies often bundle together: who generates training rollouts, how closely the teacher follows the student, and which direction of Kullback-Leibler divergence is minimized. Testing these choices independently shows that no single configuration universally maximizes acquisition of a new task while retaining pretrained capabilities.
Kim et al. theoretically compare principal component regression (PCR), gradient descent, ridge regression and a broader family of spectral estimators on individual linear-regression problems. They show that optimally tuned PCR never incurs more than a constant-factor larger risk than any monotone spectral filter under Gaussian designs, while outperforming common non-step filters by polynomial factors on selected instances.
Jin and colleagues recast existing mixture-of-experts load-balancing methods as forms of proportional, integral and derivative feedback control. Their proposed integral-derivative controller maintained more even routing across 768 experts, with its largest gains appearing when each token selected only three experts.
Smith et al. introduce BIND, an action representation for visuomotor policies that uses calibrated camera geometry to associate possible robot movements with local image features. On real-robot tasks, the authors report near-perfect success with as few as five demonstrations and substantially better robustness than coordinate-regression policies when objects or cameras move beyond the training distribution.
Hu and colleagues evaluate multimodal large language models by intervening on the prerequisite steps of composite tasks, rather than relying only on final-answer accuracy. They find that many apparent reasoning failures originate upstream, and that correcting one especially influential prerequisite often recovers most of the benefit of correcting every prerequisite.
Monteiro et al. address the growing memory cost of the key-value cache, which stores attention information for every processed token during LLM inference. Their trained selector assigns different compression choices to individual layers and contexts, achieving a 4-fold cache reduction without measured accuracy loss on models with at least 7 billion parameters.
Parikh et al. present a driving model that predicts what multiple cameras will observe, the scene’s metric depth and how the vehicle will move. Their central result is that explicitly enforcing geometric agreement between predicted depth and motion improves both planning and depth prediction, while supporting zero-shot transfer to unseen driving environments.
Kong et al. introduce TabFM, a 400 million-parameter model that treats supervised tabular prediction as an in-context learning problem. Trained only on synthetic tables generated from structural causal models, it uses labeled examples supplied at inference time to predict new rows in one forward pass, without updating its weights. The authors report leading aggregate results across all 51 datasets in the TabArena benchmark.
Madeira and colleagues address a mismatch in masked diffusion language models: training uses independently sampled masks, while generation follows sequences of states shaped by the model’s own predictions. Their PUMBA framework trains across these trajectories, carries continuous information between denoising steps and propagates gradients through multiple steps, reducing the number of model evaluations needed at matched performance.
Liu and colleagues present SPLASH, an LLM serving system that changes how attention is distributed across accelerators without draining requests or restarting workers. On B200 GPUs serving GLM-5.3, its transition-aware scheduler delivered 1.3 to 1.73 times the end-to-end throughput of deployments using a fixed layout.
Reisizadeh and colleagues argue that refusing to output targeted information is not equivalent to removing it from a language model. Across three unlearning benchmarks, they trained decoders to inspect intermediate hidden states and recovered information that standard output tests classified as forgotten. Their proposed PARS method instead trains the model to make that information difficult for adversarial probes to extract.
Kobyzev and colleagues introduce FRAC, a selective state space model built around fractional dynamics, whose influence from past inputs decays by a power law rather than exponentially. Across synthetic tasks, retrieval tests and language modeling at 1.3 billion parameters, the authors report better long-context performance than leading SSM baselines while retaining bounded-state decoding.
Kim and colleagues introduce Gumbel Straight Flow (GSF), a method for distilling sequential autoregressive generation into a continuous map that can produce tokens in parallel. Their central theoretical result is that an autoregressive model’s coupling between Gumbel noise and sampled tokens defines non-intersecting straight paths, which GSF learns to traverse in one or a few steps.
Chen, Goyal and Krishnamoorthi model two ways to scale transformers efficiently: repeatedly applying shared layers, called looping or recurrence, and activating only a subset of a larger pool of experts. Their fitted laws predict held-out loss better than earlier recurrence models and indicate that sparsity lets models benefit from additional recurrent passes for longer before returns saturate.
Wang et al. evaluate diffusion models with pass@k, the probability that at least one of k independent generations satisfies a benchmark criterion. Their experiments show that some few-step students outperform their teachers on a single draw by concentrating probability on fewer outputs, but fall behind when users can inspect many samples.
Baser et al. test whether knowledge edits and unlearning remain effective when an adversary changes how an input is divided into tokens, while preserving the underlying text. Across five language models, six datasets and six modification techniques, alternative tokenizations frequently restored the model’s pre-modification response.
Qi Pang, Virginia Smith, and Wenting Zheng propose Codetta, a steganographic protocol that allows independently deployed LLM agents to communicate covertly without a pre-shared key. The protocol achieves high capacity while maintaining provable undetectability. On three agent workloads, Codetta transmits up to 94 times more bits per visible token than the prior state-of-the-art asymmetric protocol.
The authors present HClimRep-Ocean, a machine-learning emulator that operates directly on the unstructured computational mesh of the FESOM2 ocean model. Trained on a 209-year control simulation, the model forecasts ocean currents more accurately than all reference methods at 30-day lead times, though temperature and salinity predictions are less skilled than a simple damped-anomaly persistence forecast. The authors also demonstrate competitive performance on the OceanBench benchmark against a reanalysis product.
Zhang et al. propose a framework for multi-task models that learns explicit cross-task relationships by approximating the joint distribution of task labels via pairwise interactions. Tested on YouTube's Notifications, Homepage, and Watch Next surfaces, the method improved both accuracy and user satisfaction metrics without the complexity of full joint modeling.
The authors present MW-Nowcast, a six-hour ensemble radar nowcasting model that jointly learns a deterministic predictor for organized precipitation structure and a generative flow-matching model for local residuals. The model doubles the available warning time for the most intense rainfall, delivering 6 h forecasts with skill previously confined to 3 h for the leading generative baseline.
The authors study Nash regret minimization in unknown finite matrix games with bandit payoff feedback and observed opponent actions. They develop Optimistic Payoff Balancing (OPB), which achieves instance-dependent O(log^2 T) Nash regret against arbitrary adaptive opponents, including games with nonunique equilibria. This resolves the open problem posed by Maiti et al. (2025), extending their polylogarithmic guarantee from 2x2 games to arbitrary finite dimensions.
The authors propose RefAdapt-DiT, a training-free framework that adaptively controls joint attention computation between reference images and target generation in diffusion transformers. By monitoring reference drift and target-to-reference attention, it selectively reuses stale reference states, achieving speedups of up to 3.54× on image editing with minimal quality loss.
Wang et al. introduce OracleLadder, a diagnostic tool that provides increasing levels of oracle help to large language models on multi-step math problems. By giving models a roadmap of sub-goals and their answers, they isolate failures into five categories. The dominant failure mode across all tested models is a composition gap: models can solve each intermediate step alone but cannot compose them into the correct final answer.
Schuhmann and colleagues introduce VoiceNet, a benchmark for recognizing fine-grained emotion and performance characteristics in permissively licensed, real-world speech. Their VoiceCLAP models substantially outperform existing audio-text baselines, although the evaluation measures representations and retrieval rather than conversational understanding.
The authors propose KeyRec, a training-free method that builds bounded visual memory by maintaining a dense recent cache and a structured event bank. It achieves the best compressed performance in 13 of 15 settings across four benchmarks using only 10% of the decoder-facing visual-token budget, outperforming compressed baselines by 2.21 to 18.37 points on real-time questions.
The authors introduce SAMBAR, a continual learning algorithm for vision-language-action (VLA) models that requires no replay of previously seen demonstrations. SAMBAR frames continual learning as a constrained optimization problem, solved via the method of multipliers, and selectively anchors parameters important to earlier tasks. On the LIBERO benchmark and in hardware experiments, SAMBAR retained every learned task, while all replay-free baselines fully forgot the first task they were taught.
FlashLoop is a training-free inference framework that reduces cross-loop redundancy in looped Transformers through token-sparse updates, sparse attention, and KV-residual quantization. The authors show that as loops progress, state changes concentrate on few tokens, attention-output differences concentrate on few key columns, and KV residuals become quantization-friendly. Across several looped models, FlashLoop achieves lossless accuracy with up to 1.64x speedup and 6x KV-cache memory reduction.
Gahtan et al. introduce effective depth, a scalar that treats a transformer's residual stream as a discrete-time process and measures how representation similarity decays with layer distance. Across 16 decoder-only LLMs, they find that effective depth is almost always below a closed-form reference that assumes maximally diverse orthogonal updates, suggesting that residual updates are strongly correlated.
Tran-Thien and Nock formalize mentored decoding, in which a fast draft model’s tokens are accepted while the resulting distribution remains within a chosen distance of a larger target model. Their analysis shows how this controlled deviation can increase draft acceptance and, under boosting-style conditions, produce predictions better than the target’s rather than merely approximate it.
Jerge and Jana present CANOPY, a multi-fidelity bandit algorithm for searching tree-structured candidate spaces without assuming that nearby branches always have similar value. The authors prove that its extra cost scales additively with the number of smoothness violations, and report matched-budget gains across model routing, candidate identification, test-time search, prompt trimming, and prefix caching.
Xu and Wang train a few-step image generator by first imitating a pretrained teacher, then optimizing the entire sequence of latent transitions against real ImageNet data. Their three-stage model reports FID 1.11 and Inception Score 282, while training entirely in latent space and without directly minimizing Fréchet distance.
Mukherjee and Sitaram test whether an incorrect LLM judgment reflects missing knowledge or a failure to express information already represented inside the model. Across 64 open-weight evaluators and 14 datasets, they find a substantial internal-output gap on pairwise tasks with misleading surface cues, but little benefit from internal read-outs when those cues are absent.
Lomeli and colleagues introduce model casting, a mid-training procedure that makes feed-forward network activations sparse enough for dedicated inference kernels to skip most inactive computation. They also propose LoPA Gating, which reduces the cost of identifying active units and surpasses the theoretical threefold speedup limit of conventional gated feed-forward layers.
Sarangi et al. study when finite probe evaluations of a neural network suffice for learning neural functionals. They prove identification and universality results, show hidden-layer probes are strictly more informative than final-output probes, and introduce HiddenProbe, which achieves state-of-the-art performance on MLP and Transformer benchmarks.
Wu et al. propose MaskCoFT, a fine-tuning method that uses learnable binary masks to restrict each layer's Top-K routing to a subset of experts while allowing those experts to adapt to the redirected tokens. For Mixtral-8x7B and DeepSeek-V2-Lite, the method reduces expert fetches per token by 23.7% and 10.1%, respectively, and lowers time per output token by up to 16.4% and 5.5% in a real offloading system. Average accuracy over nine benchmarks remains slightly above the base model.
Wang et al. introduce DRM, a Diffusion Reward Model that recasts reward modeling as conditional density estimation over p(r|x,y) instead of point prediction. A lightweight Diffusion Transformer denoises Gaussian noise into a reward vector with no parametric family imposed, and N samples at inference form an empirical reward distribution that can be aggregated into a scalar, variance, or quantiles. Across five benchmarks, DRM matches or surpasses baselines under matched data and backbone, and recovers multimodal structure where conventional heads collapse to a single point.