News without the noise
New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.
2566 articles · page 2 of 54 · The Research front page
The authors show that autoregressive token decoding is the primary bottleneck for trajectory prediction in compact vision-language models (VLMs) for robotic manipulation. They replace this with a geometry-compatible head that directly regresses continuous coordinate values, reducing trajectory error by over 60% and improving affordance mean average precision by 85% relative to the autoregressive baseline.
Chen compares vision-language model answers with textual personas against measured responses from people with low vision, color blindness, or hearing loss, finding the personas never outperform simple rules that use only the stated facts. The models know the effects of blur and visual illusions, but the personas do not make them simulate the actual perception.
Shu Wei and colleagues present Unlimited OCR 2, a document parsing model that selectively focuses on the pages and tokens it needs at each step. At a hundred pages, it decodes 7.0 times faster than a full-attention model while keeping per-step latency constant, and it reaches state-of-the-art accuracy on long-document benchmarks.
The authors show that image generators trained to output tokens in a fixed raster order can be adapted for any-order generation, enabling inpainting, outpainting, and editing, with only a modest amount of additional training. Their recipe outperforms MaskGIT on ImageNet inpainting and outpainting while remaining compatible with standard autoregressive inference infrastructure.
Stern and colleagues test whether training language models on structured synthetic data before ordinary text makes language learning more efficient. Across their compute-matched experiments, it does not: natural-language training performs at least as well, although synthetic warmups can help at small text budgets when given additional compute.
The authors introduce WorldFlow, a generative world model that jointly predicts future video and 3D trajectories for specified query points. Trained on 1.44 million object-level samples from simulated and real-world data, it significantly outperforms existing methods on motion forecasting benchmarks, particularly on spatial reasoning tasks where it reduces average displacement error by 54.2% relative to the best external baseline.
The authors built a professionally annotated dataset of 36 hours of user-generated videos and a reference-free evaluation framework for inline audio descriptions, rendering generated scripts to audio and measuring how well they fit natural silences. Testing nine frontier multimodal models, they find the best models describe visual events and avoid overlapping speech well, but human-written descriptions still have better narrative pacing. Ratings from blind and low-vision participants confirm the automated metrics: users narrowly prefer professional AD, yet find AI-generated AD highly beneficial for comprehension.
The authors introduce PhysAg3D, a framework that combines a 3D generative model with a vision-language agent to produce interactive assets from a single image. The agent infers semantic parts and physical priors, the generator creates a textured mesh, and the agent uses geometric and physical tools to construct joints, deformable regions, and surface bindings guided by a hierarchical interaction representation. Results on real photographs and benchmarks show plausible interactions, both rigid articulation and compliant deformation, while retaining image-aligned appearance.
Liu et al. propose cost-sensitive hinge losses for jointly training classifiers and routers in adaptive inference systems. They prove linear calibration bounds establishing Bayes consistency, and introduce a post-training procedure using unlabeled data to refine routers when labeled data is scarce.
Jeheon Woo introduces machine learning models, SwinUNETR-V2 and NeXtUNETR, based on Transformer-inspired architectures that predict electron densities with high accuracy. These models enable bypassing the iterative self-consistent field (SCF) calculation in density functional theory (DFT), achieving median energy errors as low as 0.036 meV and accurate forces for geometry optimization and molecular dynamics.
Multimodal large language models can count objects in an image but often double-count or miss items when reasoning across steps. Chen et al. show this failure is a 'reference gap', models cannot stably bind language to specific visual entities, and that adding task-appropriate external references like numbering or shading closes it, boosting accuracy by up to 25% without retraining.
Clémente and colleagues investigate why techniques for steering diffusion models can become unstable when transferred to flow-matching models, which generate outputs by progressively transforming noise into data. They identify a mismatch with the relationships learned during training and report that correcting it improves sampling quality and guidance stability across robotic control and image restoration tasks.
Xu and colleagues find that image-editing agents need more than repeated attempts: they need different tools and better evidence for judging results. Their system, EditAgent, generates alternatives with general-purpose and specialized editors, then compares them using visual evidence and automated checks.
Wang and colleagues find that a small set of attention heads helps a vision-language model select tiny details from high-resolution images, but their focus weakens as reasoning grows longer. Training these heads to attend to relevant objects raised Qwen3.5-4B’s reported accuracy from 74.1% to 76.6% across three benchmarks, without changing how the model runs at inference.
Huang et al. investigate whether chatbots can start answering questions about images and videos before processing all the visual input, without permanently losing the skipped information. Their system, Raincheck, processes that information in the background, recovering much of the answer-quality loss caused by discarding it while reducing first-response latency relative to processing everything upfront.
Randall Balestriero and Yann LeCun investigate when separately trained models learn compatible internal representations of the world. They prove a conditional result for predictive embedding models and report that a text model trained with suitable paired observations needs 42 times fewer samples than next-token prediction to reach the same alignment with vision models.
Fong and colleagues introduce Dobb, an optimizer that computes weight updates within the pieces of a model stored on separate devices, rather than exchanging information across those pieces. They report shorter optimizer calls than Muon and Dion in every measured configuration, with a roughly 13.5-fold reduction against Dion in one distributed setup.
Gupta and colleagues studied whether dialogue agents remember long conversations better by keeping two complementary kinds of records: contextual accounts of events and structured relationships between entities. Their combined system achieved 90.2% accuracy on LoCoMo and 85.5% on LongMemEval-S, outperforming the state-of-the-art memory baselines evaluated by the authors.
Chai et al. present a two-stage training framework that jointly decides which intermediate states a GUI agent should practice and how far to roll out from them. Stage I performs one-step reinforcement learning at uncertain 'pivot' states selected via action disagreement, the variance among multiple sampled GUI actions. Stage II then progressively expands to suffix and root rollouts as the policy improves. On the MobileGym and CUA-Gym-Hub benchmarks, the method reduces required environment interactions by 56–61% while achieving better task completion rates.
The authors train UI navigation agents via online reinforcement learning directly on live iOS apps. Their decoupled architecture separates planning from grounding, using an LLM judge to provide step-level dense rewards alongside task completion rewards, optimized with PPO and neighbor-conditioned value estimation (GiGPPO). The resulting 8B agent nearly matches Gemini-3.1-Pro on iOS tasks and achieves seven times the success rate of supervised fine-tuning on Android tasks.
Wang and colleagues introduce SocialArena, a benchmark that evaluates whether language models maintain a character, infer mental states and choose appropriate social responses. The authors report a consistent weakness across the models evaluated: understanding a social situation does not reliably translate into knowing how to act.
The authors present PhysicsNeMo-Mesh, a GPU-native mesh module for PyTorch that stores simulation cases with full semantic structure (named boundaries, field ranks). On one GB200 GPU, mesh operations run 6.2–1,068× faster than VTK with 72 threads, and loading a whole case is 8–88× faster. Overlapped with training, this preprocessing costs only 2.1% of throughput, enabling one dataset to serve multiple models without repeated curation.
The authors decomposed the performance gap between small and frontier language models on terminal computer tasks into three capabilities: solvability (generating a correct solution with enough attempts), verification (selecting the correct solution from candidates), and confidence (calibrating success estimates). They found that repeated sampling narrowed the solvability gap but not the verification or confidence gaps, indicating that small models need better judgment, not just better generation.
The authors systematically compared three ASR models, Phi-4, Whisper, and MMS, on six African languages under matched low-resource conditions. They found that Phi-4, a multimodal language model, achieved the best accuracy on most languages, while MMS showed comparable character error rates but saturated at less data. Domain-specific training improved Phi-4 and Whisper but hurt MMS, and using adapters from related languages outperformed training them from scratch.
The authors propose Reinforced Adversarial Learning (RAL), where a judge is trained to distinguish reference outputs from generator outputs, and the generator is reinforced to maximize the judge's belief it is a reference. Across arithmetic, code, chemistry, and scientific abstract generation, this approach matches or surpasses training with explicit verifiable rewards, even though the reward is never seen. On NIH grant abstracts, RAL produced outputs that human experts and LLM judges found harder to tell from real ones than frontier model outputs.
The authors diagnose why on-policy self-distillation (OPSD) plateaus: the self-teacher, a prompted version of the student, often produces unreliable token-level guidance. They introduce Teacher-Evolution Self-Distillation (TESD), which trains the teacher with rewards based on task correctness and context independence, and adaptively down-weights samples with unreliable teacher guidance. TESD consistently outperforms current OPSD and sequence-level reinforcement learning methods across model sizes and datasets.
Zhang et al. introduce CoCA, a closed-loop framework for rubric-based reinforcement learning in which the scoring criteria evolve along with the language model's capabilities. By contrasting rollouts generated with and without rubric guidance, the method identifies which skills are already acquired, which are elicitable with scaffolding, and which need further training. Across seven benchmarks, it reaches baseline-level performance in roughly half the training steps while improving final quality.
Shen et al. introduce Persona Learning from Feedback (PLF), which learns reusable preference-grounded assistant personas as independent LoRA adapters on a shared frozen backbone. During training, each adapter generates responses and receives feedback; a frozen self-teacher provides token-level supervision on the adapter's own prefixes. A feedback gate focuses updates on responses needing improvement. After learning, a supervised router selects or composes adapters for each request without user-specific finetuning. In experiments, PLF improves model-judged preference fit over persona prompting, persona distillation, and fixed-policy baselines in both single-turn personalization and multi-turn tutoring.
The authors present CineU, an agentic system that controls Unreal Engine via natural language instructions to construct, edit, and choreograph scenes. They evaluate on compound tasks and show that combining specialized engine knowledge (code patterns for common operations) with multi-stage verification and function-level repair substantially outperforms baselines without these components. The best configuration completes 47.8% of tasks, compared to 18.4% without knowledge and 16.8% without verification.
The authors tested two coding agents on 36 secure coding tasks with and without malicious project memory files. They found that the files changed the rate of secure completions by up to 8 percentage points but with no consistent pattern. Crucially, agents differed sharply in whether they told the developer about the file: Claude Code often cited it, while OpenCode almost never mentioned it in the final message.
The authors identify a noisy component in the gradient used for multi-token prediction in language-model pre-training. They introduce a one-line trick that removes this residual and keeps only a self-distillation signal, improving next-token perplexity, zero-shot accuracy and speculative decoding speed from 167M to 1.2B parameters with no added overhead.
The authors introduce EgoContact, a dataset of 22.5K human interaction events with pixel-precise contact annotations. They find that current affordance and vision-language models struggle to predict these contact points, often failing even to localize the object. But replacing the memory bank of an existing affordance-transfer policy with EgoContact improves grasp success from 14.2% to 31.2% in simulation and from 16% to 58% on a real robot.
Mundra and colleagues tested vision-language models on crowded scenes where identifying an object requires combining clues about its appearance, position and surroundings. The authors report substantially weaker performance than on existing benchmarks, with direct visual markers and removal of distracting objects improving results.
The authors introduce MAS-Loop, a framework that pairs an Orchestrator with a trajectory-conditioned Refiner. After each task, the Refiner inspects the execution log, localizes failures, and proposes bounded patches to agents, information flow, or execution configuration. The updated system is re-executed on the same task, and the score difference provides learning signals for both policies, enabling co-evolution. Experiments show consistent improvements across interaction, multi-hop reasoning, and general reasoning benchmarks, and the resulting dataset supports improved test-time MAS selection.
Suhas et al. at the Max Planck Institute for Informatics introduce OneScene-Bench, a human-annotated benchmark that forces AI vision models to integrate evidence across multiple views rather than process each image independently. Testing twelve open-weight and proprietary models, the authors find that unique-object counting collapses when views are complementary, and that contradiction detection hovers at chance, while spatial reasoning is far stronger.
The authors systematically varied where indirect prompt injections appear in the environments of AI agents, keeping the injected content fixed. They found that agents follow unsafe instructions far more often when placed in documentation files than in code or data, up to 2.6 times on average. They call this uneven susceptibility the trust region effect.
The authors propose Adversarial Self-Distillation (ASD), which uses the model's own safe behavior on clean prompts as a teacher signal for the same model under adversarial attack. They find that ASD reduces attack success below that of a state-of-the-art defense and teaches the model to recover a refusal even after it has started a compliant response.
The authors present a continuous video super-resolution method that fuses low-resolution frames directly onto a high-resolution grid at their predicted sub-pixel motion positions, unifying temporal interpolation and upscaling into a single operation. On GoPro and Adobe240 benchmarks, it improves reconstruction fidelity and perceptual quality over the state of the art while running 6 to 11 times faster.
The authors demonstrate that the evaluation of single-view novel view synthesis methods is flawed due to misaligned coordinate frames. They propose using 2D-to-2D and 2D-to-3D matching to properly align reconstructions, revealing that explicit 3D Gaussian methods outperform video-based ones, contrary to prior reports. A simple baseline using off-the-shelf video generation and geometry estimation achieves comparable performance without task-specific training.
Shen and colleagues studied how language-model agents manage planning, reasoning and verification during extended tasks. Verification was most strongly associated with success, but often happened late; their proposed coordination system keeps goals explicit and checks intermediate results, improving performance on two task benchmarks.
Ranjan and colleagues investigate why checking several proposed tokens together speeds up some sparse language models more than existing cost models predict. They find that neighboring tokens reuse expert subnetworks, while hardware executes those subnetworks more efficiently than their weight sizes alone would suggest.
Wu and colleagues show that weakening a copy of a reasoning model can provide a useful signal for improving the original. Across 25 model–benchmark settings, their two approaches increased average accuracy by 4.0 and 5.5 percentage points without requiring reference answers, expert demonstrations or environment feedback.
Jonathan F. Jao and Andrew Kyle Lampinen show that a language model’s internal activity can help predict which conversations will lead to problematic behavior later. Using these predictions to guide automated safety audits, they obtained higher peak judge scores across 18 target behaviors and three model sizes.
Sener and colleagues combine a generative weather model with reinforcement learning to improve rainfall predictions without smoothing away intense, localized storms. They report competitive or better forecast accuracy through 24 hours, alongside realistic spatial patterns, including signals of extreme accumulations in two flood case studies.
The authors train a student language model on its own responses using token-level feedback from a stronger teacher, a setup known as on-policy distillation. To stop the student's reasoning from drifting as responses grow long, they replace each response's earliest high-disagreement segment with a short teacher-written segment, then let the student continue. This stabilizes training and improves student-only accuracy across five mathematical reasoning benchmarks and two teacher-student pairs.
Shbita et al. present LLMON, a tagging framework that separates executable instructions from passive data in AI language models. After post-training Granite-4.1 base models at 3B and 8B scale on a 4.6 billion token structured corpus with supervised fine-tuning and reinforcement learning, indirect prompt-injection attacks succeeded in only 0.02-5.6% of cases, down from 53-100%, with general capabilities mostly intact.
Shuo Wen and colleagues built The Prestige, a benchmark of 2,177 open-ended questions grounded in professional magic performances, and tested 14 leading vision-language models against human baselines. The best model scored 47.08%, versus 92.79% for humans, a 45.71 percentage-point gap. Models repeatedly failed to recall objects absent from the current frame, missed surprises that humans noticed, and struggled to reject false premises.
The authors present Metatron, a system that separates whole-body motion execution from task intent. A frozen base policy trained on motion capture and synthetic data accepts partial goals (prompts) to coordinate movements. Programs called skills translate task intent into prompts, enabling new behaviors to be added through code without retraining the policy.