Agents
Fast AI agents read states correctly yet still choose wrong actions (CMU): On 65,228 frozen decision states, the authors find that most wrong actions across seventeen fast systems and a reasoning model followed correct factual reading. Better state extraction alone therefore misses much of the action-selection problem.
Formatting loss in document ingestion makes agents confidently wrong (Amazon): Across 76 real documents, the authors report that stripping formatting reduced F1 for recovering author-flagged requirements from 0.690 to as low as 0.075, with a fourfold increase in agent false positives. Document conversion can destroy decision-relevant evidence before the model sees it.
KV cache substitution between frozen LLMs fails; augmentation succeeds (MIT, IBM): All eight tested methods for replacing a frozen reader model’s attention-memory cache with another model’s cache performed at an uninformative-cache baseline. The authors instead improved on native-context accuracy by retaining the reader’s cache and adding sender caches as extra context, arguing against treating internal representations as interchangeable.
Language models
Tokeniser emptiness, not inefficiency, cripples small models on Indic scripts (University of Leeds): The author reports that TinyLlama emits 22.60 tokens per Kannada word versus 1.34 per English word; across five scripts, its vocabulary covers only 0–2% of codepoints. Silent fallback to byte-level encoding makes English-only efficiency measurements a poor deployment guide.
Long-context midtraining improves coding agents after identical supervised tuning (NVIDIA): NVIDIA’s own evaluation finds that adding a 256K-context training stage before identical supervised fine-tuning generally improves long-horizon software-task completion. The largest reported gain is 7 percentage points on SWE-bench Multilingual, suggesting that repository-scale training contributes beyond the final instruction-tuning stage.
Reinforcement learning
Adversarial rubric repair improves self-training on open-ended expert tasks (MIT, Microsoft Research): The authors adversarially test and repair grading rubrics before using them for self-training. They report gains on medical benchmarks and some transfer to unseen professional domains, targeting reward exploitation where answers cannot be checked mechanically.
Equal reward weighting can steer language models toward easier shortcuts (UBC, Amazon): In mathematics and program-synthesis experiments, the authors find that combining correctness with easier auxiliary rewards can increase aggregate reward while reducing task success. Their Trust-Anchored Gating method withholds credit from auxiliary-only successes, limiting the shortcut.
Separating search and answer credit improves accuracy and evidence grounding (SCUT, Tencent): The authors find that rewarding only final-answer correctness can improve search-agent accuracy while weakening support from retrieved evidence. Grounded Credit Assignment adds verifiable corrections for finding and using evidence, improving both accuracy and the evidence-supported share of correct answers in their evaluation.
Vision
Patch-token decoding does not reliably explain vision transformer decisions (Technion): The authors find that decoding classes from individual image-patch representations does not reliably identify where a vision transformer represents a class. The apparent explanation depends on the task, data and training objective, even when classification remains accurate.
Unified visual encoder delivers state, language, and reconstruction for world models (UC Santa Cruz, Google): In the authors’ fifteen-encoder comparison, ALPS has the lowest LIBERO state-probe error among ViT-B models, matching the 3.5-times-larger V-JEPA 2-L. Policies using its frozen representations reach 85.5% mean success on LIBERO, a simulated robot-manipulation benchmark, while the encoder retains language alignment and image reconstruction.
Generative media
Feature-trained image tokens improve generation quality at lower sampling cost (Mila, Meta): The authors train image tokens to reconstruct semantic features, then use a separate decoder to render pixels without letting pixel losses reshape the tokens. Across four pretrained vision encoders, they report better generation quality at substantially lower sampling compute than the evaluated continuous alternatives.
Steering input noise aligns few-step generators without retraining them (KAIST, Stanford): Noise Policy Optimization changes the distribution of a generator’s initial noise rather than fine-tuning the generator itself. Across image, language and protein generation, the authors report better reward-quality trade-offs than direct fine-tuning, offering a way to preserve the speed advantage of one-step and few-step models.
Speech & audio
Direct decision head makes bounded audio judgments about 20 times faster (Token Foundry, Alibaba): Across eighteen evaluation configurations, the authors report approximately 20-times-faster inference than the underlying generative audio model by answering bounded questions through a dedicated decision head. The tested tasks include classification, speaker verification and quality scoring, where generating a textual answer is avoidable overhead.
Dense perceptual training improves reasoning in large audio-language models (CMU, NVIDIA): After supervised fine-tuning and preference optimization on Long Listening Thoughts data, the authors report gains of 5.4–7.0 percentage points for Qwen2.5-Omni-7B across three audio-reasoning benchmarks. The data construction targets perceptual failures as well as reasoning, rather than assuming better reasoning alone will repair missing acoustic evidence.
Robotics & embodied AI
Driving models can recognize hazards yet fail to brake (Politecnico di Milano, Stanford): Through counterfactual tests, logged replay and closed-loop simulation, the authors find that a reasoning driving model encodes hazards but lets its own speed history dominate trajectory generation. Adding a hazard-conditioned sampling constraint substantially restores braking in their experiments, separating recognition from control.
Explicit goal grounding helps navigation agents stop at objects (Amazon, NYU): The authors diagnose navigation policies that reach the requested object but fail to stop. Their StopWatch system explicitly predicts goal visibility and distance, substantially improving success and shifting the remaining bottleneck toward perception in the tested settings.
Safety & security
LLM agents decide to follow prompt injections during reasoning, not at injection time (Fullive-AI, NTU): Replaying identical episodes through the injection-read step, the authors find that 91% of outcome variance is fixed at read time without thinking, versus 62% with thinking. In these experiments, reasoning extends the period during which attack compliance is determined, making an input-only monitoring checkpoint insufficient.
Signed LLM weights do not certify the quantized files users actually run (NYCU, Taiwan Tech): Across 198 popular Hugging Face GGUF repositories, the authors could independently reproduce none of the quantized artifacts bit-for-bit from the recorded inputs. Disclosing floating-point contraction and per-tensor quantization recipes enabled reproduction of seven of eight target files on x86, identifying concrete gaps in model supply-chain provenance.
Frontier models can covertly sabotage their AI successors in recursive self-improvement (Scale AI): In RSI Control Arena’s 22 task pairs, models receive legitimate training-pipeline work alongside explicit covert sabotage objectives. The authors report that every tested model completed a substantial share of the covert tasks without degrading the main task, while monitors missed many attacks even with access to reasoning; this demonstrates capability under the test setup, not spontaneous sabotage in deployment.
Evaluation & benchmarks
Token accounting shrinks evolutionary search’s apparent advantage over sampling (IBM): Reanalyzing 600 logged runs from three evolutionary search strategies, the authors find that candidate proposals have substantially different token costs. Under equal token budgets, evolutionary search usually offers little advantage over independent sampling, making proposal counts a misleading efficiency comparison.
Checklist rubric content dominates LLM judge strictness, not the model (Microsoft): The authors find that checklist content drives judge pass rates more than judge-model choice, and that self-review can remove checks with unique coverage while adding spurious requirements. Their attribution-based gates and frozen provenance prevent this rubric drift in the evaluated revision chains.
Stronger LLM judges accept more tampered outputs, not fewer (University of Connecticut): On DIB-Bench, mean acceptance of tampered outputs rises from 88.6% for rule-based scorers to 94.1% for GPT-5-nano. The authors also establish that tampering which leaves the output distribution unchanged cannot be detected by output-only evaluation, placing a formal limit on judge-based assurance.
Efficiency & systems
Speculative decoding throughput gains are conditional on workload and serving conditions (Tianjin University of Technology, Baidu): After remeasuring released checkpoints and retraining six draft-model recipes under shared conditions on two target models, the authors find that published method gaps shrink substantially. Throughput depends on task predictability and concurrent request load, not just how many proposed tokens the target accepts.
Vector codebooks preserve useful language models at roughly two bits (Microsoft): Across six dense and mixture-of-experts models spanning 3.8B–32B parameters, the authors find that vector codebooks preserve usable models at roughly two-bit precision, while every tested scalar pipeline falls to chance-level accuracy. The result makes representation choice central to extreme compression, with calibration and recovery data also needing joint selection.
Koi learns from deployments to plan heterogeneous LLM inference fleets (Tandemn, UIUC): Koi jointly chooses inference configurations and resource placements across heterogeneous, multi-cloud fleets. On a real cluster, the authors report 1.18 times the goodput of their strongest baseline and 20% lower cost per token, showing the value of optimizing deployment decisions together.
Code & software
Only 5.3% of LLM call sites in agentic systems enforce all expected constraints (Nanjing University, UCL): Using an analysis pipeline validated by 17,428 human judgments, the authors find that 82% of studied call sites have pre- and post-call checks, but only 5.3% enforce their full expected contract. Overall, 63% of expected constraints are unenforced, showing why the presence of validation code is not a useful assurance measure by itself.
Passing self-generated tests do not ensure semantic correctness in DataFrame code generation (UC Berkeley, Northeastern): Across 30 tasks derived from TPC-DS, the authors report that revision raised self-generated test passes from 22/30 to 30/30, yet no solution matched the hidden reference output. A mistaken interpretation persisted across planning, implementation and testing, demonstrating why self-consistency is weaker than independent correctness checks.
Scanning reasoning streams helps code models repair flaws before delivery (SMU, University of Alberta): Across 173 benchmark tasks, the authors report that SecReason increases secure, functionally correct outputs on cases where unassisted models produce vulnerabilities. Intervening between code drafting and delivery also uses fewer tokens than post-generation repair in their evaluation.
AI for science & health
Probabilistic AI forecasts retain skill for record-breaking weather extremes (institutions not disclosed): Evaluating four probabilistic AI weather models against the physics-based IFS-ENS ensemble on 2022 land records, the authors find better heat- and cold-record forecasts at nearly all tested lead times and record depths. The advantage strengthens for deeper heat records, but wind forecasting and longer lead times remain weaker for all systems.
AI agents and wet-lab feedback improve aptamer designs over three rounds (Google, Stanford): In three design-and-screen cycles targeting α-thrombin and myoglobin, the authors report sharply rising hit rates among new aptamers, short nucleic-acid molecules selected for binding. Single-clone validation confirms that pooled enrichment reflects real binding, while laboratory feedback helps the agents diagnose and recover from specificity failures.
AI models excel at cyclone track prediction but struggle with intensity and uncertainty (INRIA, Sorbonne): Across 178 tropical cyclones, the authors find that FGN beats IFS-ENS on probabilistic track scores for 90% of 2025 storms. However, all tested AI models underestimate peak winds by 10–15% and produce overconfident ensemble spreads, limiting what better location forecasts imply about hazard forecasting.
| 