News without the noise
New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.
2566 articles · page 3 of 54 · The Research front page
Li and Ni examine when measurements derived from trained models can be corrected for randomness or compared fairly. They find that a textbook noise correction can fail mathematically, while averaging benchmark results across settings can conceal statistically supported reversals in which model leads.
The authors propose Seq@K, a metric that gauges a language model's ability to solve a task within K attempts when given feedback after each failure. Evaluating 16 frontier models on 7 benchmarks, they find that Seq@K rankings often diverge from standard Pass@K rankings, and cost-performance Pareto frontiers shift. This suggests that choosing a model based on feedback-blind accuracy may select the wrong one for iterative agentic workflows.
The paper studies building prediction intervals for regression when test inputs follow a different distribution than calibration data, a setting known as covariate shift, while the response mechanism stays the same. The author proves that one rule, using the inverse conditional density as a score with randomized weighted conformal calibration, has minimum expected interval length under the target distribution among all valid rules, for any fixed calibration size. Simulations and four regression benchmarks show it often yields shorter intervals than existing weighted methods.
The authors show that a jailbreak embedded in a Markdown skill file, loaded as trusted configuration, can bypass safety training in AI agents. Across seven open-weight models and 270 forbidden questions, mean attack success rose from 0.5% under a benign prompt to 45% with the poisoned file. Obfuscated payloads did not improve the attack.
The authors analyzed 1.53 million AI agent skills from GitSkills, developing a codebook and a trained detector to identify oversight-loosening directives. They found that approximately one in seventeen skills (about 88,000) contains such a directive, most commonly through permission flags that tell the agent to proceed without approval. Loosening directives are more likely in skills that run shell code or install packages, and they propagate widely through copying from a small number of popular parent skills.
Training across many graphics processors requires repeatedly gathering model weights and combining gradients, the updates calculated during learning. Wang and colleagues show that making these transfers read and write the model’s existing storage directly can reduce training-step time by up to 27.1% against standard PyTorch FSDP2, or 6.0% against its symmetric-memory variant.
Wang et al. tested five large language model families and four deliberation topologies on 120 real emergency department cases, swapping only demographic labels to create counterfactual versions of the same patient. No team configuration neutralized the resulting triage disparity, and late fairness fixes failed. Only an ensemble that makes both patient versions visible to all agents and pools them into the final decision reduced the gap, by 67 to 94%.
The authors show that collaborative decoding, a common design for private-context AI assistants, leaks information from private documents through messages exchanged with the cloud, even when the answer itself seems clean. They propose ADLER, a decoding protocol that guarantees the cloud receives nothing beyond the answer the user sees, and a leakage ledger that budgets how much the answer can reveal about private data.
The authors introduce SciGame, a benchmark that converts scientific laws into paired original and counterfactual environments. Testing five frontier models, they find that agents' familiarity with a law hurts performance when the underlying relation is altered, and that adaptive experimentation benefits only the strongest models. Even when agents provide good explanations, they frequently fail to recover the exact symbolic form of the law.
Zhu et al. present a way to compress the intermediate representations language models retain while generating long answers. Their approach reduces the work needed to encode each new representation and adapts its compression dictionary without enlarging it, with reasoning and long-context evaluations reporting a favorable quality-efficiency trade-off.
Tu and colleagues present GOFish, a method that turns calibrated fisheye photographs into a renderable virtual scene and a matching surface mesh for collision simulation. The authors report better image quality than prior methods on wide-angle benchmarks, while preserving peripheral views that earlier Gaussian rendering methods could not fully cover.
NVIDIA’s authors present two models designed to combine reasoning and tool use with efficient deployment. The larger model, Nemotron 3.5 Super, scores 51.78% on SWE-bench Pro, exceeding its larger predecessor with substantially fewer parameters active during computation. Separate experiments show benefits from software-engineering training, parallel agents and low-precision inference.
Ganguly and colleagues combine an off-the-shelf language model with a parallel proof-search system, achieving near-complete coverage of several mathematics and code-verification benchmarks. They also generate harder verification tasks from competitive-programming implementations, where the system proves 1,054 of 2,000 sampled statements within a 30-minute budget per statement.
Hu and colleagues trained language models from scratch with deliberately poisoned data, then turned them into command-line agents using only clean post-training data. The resulting agents could execute an attacker-chosen command when a trigger appeared, showing that benign agent training does not necessarily remove backdoors inherited from pretraining.
On three contact-rich tasks with a Unitree G1 humanoid, FLoRe's authors report 80% success with force-reactive control versus 26.7% with a position-only policy. Elsewhere, the week's strongest evidence challenges familiar shortcuts: task reward does not establish world-model fidelity, distrust does not prevent harmful tool calls, and estimated computation does not establish deployment speed.
The authors propose StruPO, a prompt editor that uses calibrated scene graphs to condition and supervise editing decisions. Across multiple test sets and generation backbones, StruPO improves the faithfulness-aesthetics trade-off, especially for complex relational constraints. The policy requires only one pass after graph calibration, without needing iterative critique-rewrite loops.
The authors introduce the Control Map, an evaluation framework for test-time model updates that records the ordered updates completed before each scored answer. At a 760M-parameter checkpoint, 24 schedules with the same update count produced accuracy ranging from 19.7% to 34.2%, and no editor dominated across models and benchmarks.
The authors introduce PROBE, a benchmark of 2,457 bugs that maintainers fixed over 18 months across 20 GitHub repositories. Evaluating ten frontier coding agents, they find recall below 21% and precision below 9% against this reference set. Most agent findings are valid bugs, but they match other agents' findings far more than they match maintainers' fixes.
The authors propose EMMA (structured Episodic Memory for Multi-Agent workflow synthesis), which treats the entire configuration of a multi-agent system (controller routing, agent prompts, procedural skills) as episodic memory. EMMA introduces explicit scope assignment and backbone-independent guards for each update, leading to consistent improvements over strong baselines across five backbones and three benchmarks, with up to 32.5% performance gain and 17–38% fewer training executions.
Chen et al. present GateRQ, a compression method for the large lookup tables that supply factual information to some language models. Without retraining, it reduces GPU table memory 7.8-fold on DeepSeek-V4.1-Flash and increases serving throughput by 11% to 31%, while their evaluations report small losses on PopQA, a benchmark targeting facts about uncommon entities.
Hu and colleagues study an attack that inserts instructions into tool descriptions read by AI agents using the Model Context Protocol (MCP). Their Log-To-Leak attack adds an attacker-controlled logging call after legitimate tool use, showing that successful task completion can coexist with unauthorized disclosure of interaction data.
Lin et al. introduce MVRCap, a reinforcement learning framework that trains video captioning models using a hierarchical multi-dimensional verifiable reward mechanism. The authors report state-of-the-art performance on the VDC, CAPability and VC4VG benchmarks, showing that structured verification improves both comprehensiveness and factual consistency of generated captions.
The authors propose API-TAIL, a text-only audit that detects when an LLM API secretly routes some requests to a cheaper substitute. It scores each response under the claimed model, aligns one effective temperature to absorb batch-level shifts from legitimate decoding choices, then flags endpoints with too many remaining tail responses. On real OpenRouter endpoints it kept false alarms low while detecting 10% substitution in 13 of 16 substitute settings with 200 requests.
The authors propose a structural approach to impact analysis for LLM-driven software evolution. They model design assets as a knowledge graph and Merkle DAG to compute causal chains of changes, then inject these as deterministic context for agents. Early deployment on a large payment platform shows that this method prevents under- and over-modification.
The authors introduce CoHARL, a population-based reinforcement learning framework that co-evolves an LLM agent's policy and its executable harness, the code that controls tool use and feedback. On SWE-rebench, CoHARL achieved a 27.6% mean resolved rate, outperforming harness-only search by 6.4 percentage points and policy-only training by 7.3 points.
The authors propose CASEL, a runtime layer that maps proposed tool calls to one of six intervention modes, balancing failure probability, consequence, reversibility, ambiguity, and evidence staleness. In a 20-seed simulation with 120,000 test actions per seed, CASEL roughly halved harmful-side-effect trajectories relative to an uncertainty-ranked gate, at equal compute and with the same intervention menu. A simpler risk-ranked gate achieved slightly lower harm but required 45 percent more human approvals.
Tartaglini et al. tested whether vision-language models (VLMs) share human visual communication abilities by probing how simple symbols affect model behavior on unrelated tasks. They found that a red heart strongly steers models toward recursive solutions on simple coding problems, and this influence is absent from the model's explicit reasoning. Further experiments in communication games revealed a gap between how models generate and interpret visual meaning, and their visual associations only partially align with human intuition.
The authors propose SpeeDiff, a latent diffusion model that jointly trains the VAE and diffusion model from scratch. The key innovation is a Tweedie Pixel Reconstruction (TPR) loss that provides pixel-level feedback from intermediate noisy latents, preventing latent space collapse. On ImageNet 256×256, SpeeDiff-XL achieves an FID of 1.50 without guidance and trains 140× faster than Vanilla SiT.
Joseph and Kulesza introduce OptimizeRC, a differentially private algorithm that selects an L2 clipping bound for vector sums by optimizing residual coherence, the directional alignment of vectors exceeding a candidate bound. They prove that residual coherence closely tracks bias for heavy-tailed data, and show empirically that their method reduces error compared to quantile-based baselines across multiple real-world datasets.
The authors simulate LLM agent societies and compare push messaging, where every message enters context on arrival, with pull inboxes, where agents set their own reading rules. Push fails under concurrent load, while an inbox prioritizing deposit-bearing messages recovers about 60% of needed agents under heavy spam.
The authors show that linear steering vectors extracted from either text or image representations causally transfer zero-shot to the other modality in unified autoregressive models. Cross-modal steering recovers 84% of intra-modality steering effect from text to images and 74% from images to text in UniAR, with similar or better transfer in Emu3.5 and Liquid. Even modality-specific concepts like formal language induce consistent visual effects (e.g. formal clothing) when transferred.
Wang et al. introduce GUI-JEPA, the first GUI agent policy built on a Joint-Embedding Predictive Architecture. It replaces token-by-token action decoding with a single batched pass that scores candidate actions in latent space, achieving 2.8-4.0x speedup over a fine-tuned Qwen2.5-VL-7B agent with only 6% of its parameters.
Cao et al. introduce Evolving Agent Teams, a multi-agent framework in which a central supervisor monitors progress, diagnoses why a team is stuck, and restructures the team's roles, all grounded in a persistent file system. On four CUDA kernel optimization tasks, the framework outperforms strong single-agent and static-team baselines under matched token budgets, and with extra budget surpasses the production FlashInfer MLA decode kernel by 1.21x mean speedup.
The authors propose evaluating GUI grounder clicks by whether they produce the same outcome as clicking the annotated target, rather than whether they land inside a bounding box. Using an automated probe that executes and fingerprints clicks on live web applications, they find that on a code forge and an online shop, 17.7% and 31.2% of clicks rejected by box scoring are actually outcome-correct. A blind audit of 11 models on ScreenSpot-v2 confirms the same pattern, with corrections of 0.2 to 1.8 points, as large as the gaps typically reported between models.
The authors introduce TAPESTRY, a method that repurposes a pretrained video diffusion transformer into a robotic action policy by treating robot actions as one-patch tokens in the VAE latent space. The shared backbone is trained on three diffusion objectives, and at test time only the action pathway is used, producing action chunks in a single DDIM step at 84 ms per chunk.
The authors show that existing gradient-based data attribution methods for instruction-tuning large language models are sensitive to minor changes in wording or formatting, even when the content is unchanged. They propose Content-Invariant Gradient Attribution (CIGA), which selects an influential and stable layer for gradient features and fuses them with semantic representations, improving selection consistency and downstream performance.
Bhatia et al. introduce PRISM-BENCH, a benchmark of 1,000 questions with human-written reasoning traces that enable diagnostics of perception, reasoning content, and efficiency. Their perception probe attributes about 85% of frontier-model failures to misperception rather than faulty inference, and a trace-alignment measure predicts correctness across 25 systems with macro AUROC 0.66.
The authors propose NLD-Uni, a unified discrete diffusion language model for visual understanding and generation. They introduce cross-paradigm on-policy distillation to transfer skills from autoregressive VLMs and continuous flow models, along with a progressive self-refining diffusion process that improves image quality at inference. NLD-Uni achieves state-of-the-art results among unified DLLMs on question-answering, grounding, generation, and editing.
Salapa introduces kw-bench, a benchmark built from real VAT reconstruction engagements, testing four agent harnesses on DeepSeek v4.1 Flash and Claude Opus 5.5. On DeepSeek v4.1 Flash, only the independent harnesses OpenCode and micro-cc succeeded, scoring 0.542 and 0.528; the lab harnesses Claude Code and Codex scored 0.00 each. On Claude Opus 5.5, micro-cc and Claude Code achieved 0.9986 and 1.000 in 22 and 25 minutes, collapsing a weeks-long engagement to minutes.
The authors propose BoxWorld, a framework for real-time interactive video worlds where user-defined rules control object behaviors. They separate interaction reasoning from generative rendering: a coding agent creates a Box World Engine that applies rules and updates a minimal object-centric state interface, while a Box-State Generative Renderer autoregressively renders the state stream into video. BoxWorld outperforms end-to-end world models across multiple benchmarks and enables fine-grained control over object motion, state switches, and conditional events.
Qiu et al. present ActiveAlign, a system that uses robot movement and touch to improve correspondence between the physical world and a simulation. The authors report millimeter-level object alignment and demonstrate fine manipulation on a Unitree G1 robot, including pressing small buttons and moving sliders.
Wang et al. introduce Policy-Compatible Reinforcement Learning (PICO), a training method for reasoning problems on which every sampled answer fails. Rather than directly copying expert solutions, it uses expert-guided decoding constrained by the student model’s own probabilities, recovering verified successes that can supply otherwise missing learning signals.
Ospanov and colleagues test whether large language models can detect flaws and award appropriate partial credit in Olympiad-level mathematical proofs. Their benchmark exposes continuing disagreement with expert graders, while their proposed MABGrader method improves agreement over ProofGrader, a leading comparison system.
Wang et al. reduce the random-number generation needed for INT8 language-model training by sharing rounding thresholds across groups of gradient values. Their error-corrected method matches the validation loss of independent stochastic rounding in the reported pre-training experiments and runs full dense-model training steps 1.24 to 1.32 times as fast as BF16.
Wu and colleagues report that updating a language model equally on every chunk of a long context can dilute information needed to answer a particular question. Their Select-TTT method instead gives query-relevant chunks stronger updates, consistently outperforming uniform updates and zero-shot baselines in the reported evaluations.
Leng and colleagues study a conflict in reinforcement learning: rewards for properties such as response length can undermine the reward for actually completing a task. Their CONCUR method adjusts auxiliary scores to respect task success, with both variants exceeding GDPO by more than four points on the paper’s benchmark average for each of three models.
Chen and colleagues present Hy-Nomas, a video world model that generates successive scenes in response to free-form text instructions and continuous camera controls. Its central design keeps descriptive context available while limiting action instructions to their intended intervals. The authors report better human-evaluation results than existing world models, although the supplied material gives no scores or evaluation sample sizes.
Deng and colleagues introduce InfraDev, a benchmark asking coding agents to reproduce substantial changes to software for AI training, inference and reinforcement learning. Their evaluation of six frontier agents finds that passing existing tests often fails to establish correctness, and that incomplete task descriptions can also prevent successful solutions.