Sat 31 Oct
LiveGet it free
ZOTPAPER

News without the noise

Research

New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.

2566 articles · page 18 of 54 · The Research front page

AllAgentsLanguage modelsReinforcement learningVisionGenerative mediaSpeech & audioRobotics & embodied AISafety & securityEvaluation & benchmarksEfficiency & systemsCode & softwareAI for science & healthTheory
Viewgridlistcompact
Vision

Motion capture from video works with any target skeleton topology

The authors present CAST, a framework that recovers motion from a single monocular video onto a user-supplied target skeleton, even when that skeleton's topology—joint count, branching, and rest pose—differs from the subject in the video. It introduces GRACE, a module that corrects parent-to-child rotation propagation using reliability-gated alignment with predicted global bone directions. On matched-rig and cross-skeleton benchmarks, CAST reduces MPJPE by 27.6–62.5% and 54.7% respectively over a strong baseline, and a causal streaming variant exceeds 51 FPS.

Evaluation & benchmarks

Survival sandbox benchmark evaluates autonomous agents on sustained goal execution

The authors introduce ASIWorld, a survival sandbox with persistent world and character state, character-scoped observations, and a programmable asynchronous interface. They evaluated nine agent configurations on two tasks, production and exploration. Eight configurations completed the production task and six completed both tasks, but analysis of execution trajectories revealed significant differences in preparation, combat, and recovery behaviors that endpoint success alone did not capture.

Language models

Self-improving data synthesis adapts embedding models to new tasks with minimal human input

The authors propose SYNTRA, a method that generates task-specific training data for embedding models by iteratively improving synthesis instructions using feedback on trial outputs. Evaluated on 24 retrieval datasets across 15 domains, SYNTRA-generated data improved retrieval performance over an MS MARCO-trained baseline on all tasks, and automatic self-improvement outperformed human calibration on four of six tasks.

Robotics & embodied AI

DexCast learns real-world dexterous control from human videos without teleoperation

Gao, Zhang, Wang and colleagues present DexCast, a framework that transfers dexterous manipulation skills to a five-fingered robot hand using only monocular human RGB-D videos. The authors show that combining 4D hand-object reconstructions with a diffusion policy conditioned on a pretrained hand-motion perception module removes the need for teleoperated demonstrations and supports zero-shot cross-scene deployment.

Robotics & embodied AI

Context scheduling improves long-horizon robot video prediction

Hao Wu and colleagues from multiple institutions introduce ReCAP, a training-free context-scheduling method for autoregressive robot video world models. By preserving the initial scene anchor and a contiguous window of recent visual-action blocks while evicting older generated history, ReCAP improves visual fidelity, action consistency, and computational efficiency across four robotics benchmarks, reducing LPIPS by 63.3% on BridgeV2 and KV cache by 73.3%.

Evaluation & benchmarks

AI agents still cannot reliably deliver end-to-end on underspecified business requests.

The authors present FDE-Bench, a benchmark of 49 cases built from real customer projects. Agents must clarify underspecified requirements with a simulated customer, then iteratively optimize a solution until it meets acceptance criteria. Across 18 agent configurations, the best mean delivery score was 0.5510 against a customer-accepted reference of 1.0, and the best delivery success rate was 6.12%.

Agents

Co-Adaptive Routing and Retrieval Learns Query Rules Without Supervision

Qi et al. propose Co-Adaptive Rule Routing and Constraining Retrieval (CoRE), a framework that jointly learns which reasoning rule to apply to a query and how to retrieve evidence satisfying that rule, without explicit query-rule supervision. On STaRK-QA benchmarks across scientific discovery, biomedicine, and e-commerce, CoRE achieves the strongest average retrieval performance, and ablations confirm that co-adapting the router and retriever is essential.

Robotics & embodied AI

Cross-video human demonstration pairs improve robot world-action model transfer

The authors introduce WATCH, a cross-video pre-training objective that learns transferable world-action representations from task-paired human demonstrations. By conditioning on one demonstration and predicting another's actions and future frames, the model learns task-relevant interactions that transfer across scenes and embodiments. After robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy, with gains largest on unseen levels.

Efficiency & systems

Echo enables fast 4-bit tensor-parallel all-reduce with controlled error accumulation

The authors present Echo, a 4-bit all-reduce for tensor parallelism that bounds quantization error accumulation across layers. Echo moves data in 4 bits but sums in high precision, and feeds back each rounding error to cancel in later calls. On A100 GPUs, it achieves up to 2.09x speedup over 16-bit NCCL all-reduce, with average accuracy within 0.4 points of 16-bit inference on Qwen3-8B and final pretraining loss within 0.9% of 16-bit.

Reinforcement learning

RL gains in no-think mode stem from reactivating existing reasoning

The authors trained Qwen3-1.7B, 4B, and 8B models with GRPO in no-think mode on the DAPO-Math-17k dataset. As accuracy improved, responses grew longer and more reflective, and internal activations shifted toward the model's think mode. Causal interventions showed that replacing a low-dimensional think-related activation subspace removed 75% of the RL accuracy gain, establishing that much of the improvement comes from reactivating existing reasoning rather than learning new strategies.

Theory

Polynomial-time algorithms face fundamental limits in online convex optimization with switching experts

Assos et al. study the problem of competing with a comparator that switches among a small set of experts in online convex optimization. They prove that under standard hardness assumptions, polynomial-time algorithms cannot achieve the information-theoretic lower bound of O(k) regret, instead suffering at least (k√d)^{1-o(1)} regret. This establishes a separation between what is possible in principle and what is computationally feasible.

Language models

Entropy signals guide efficient reasoning repair in LLMs

The authors find that incorrect reasoning traces in large language models have mean token entropy roughly 112% higher than correct ones, a signal that can be used to localize errors. They propose REWIND, a controller that identifies high-entropy peaks, backtracks to the largest entropy gradient increase, and regenerates the suffix from that point. On five benchmarks and three model families, REWIND outperforms random-proposal MCMC methods by 0.51–10.00 percentage points without additional training.

Robotics & embodied AI

Systematic study reveals what matters for world modeling in robot VLAs

The authors present the first systematic, controlled empirical investigation of world modeling in Vision-Language-Action models (VLAs). By categorizing existing approaches into four integration paradigms and evaluating them with diverse world representations and policy backbones, they find that 2D semantic representations (DINOv3, SigLIP) provide consistent improvements, the decoupled MoT Future Expert paradigm offers strong out-of-domain robustness, and world modeling yields clear benefits on long-horizon and dynamic tasks.

Vision

Generative framework converts sparse panoramas into annotated multi-room 3D scenes

The authors present Holi-Spatial 360, a pipeline that transforms sparse 360-degree panoramas from real-estate listings into photorealistic, semantically annotated 3D environments. Key innovations include Correspondence-Guided Geometric Optimization to establish reliable cross-view geometry and adapted generative priors for view densification. The resulting Holi360 dataset, with over 1,000 scenes and 2 million spatial question-answer pairs, substantially improves the spatial reasoning capabilities of fine-tuned vision-language models.

Speech & audio

Post-training gains in audio models may reflect format adaptation, not true audio understanding.

The authors investigated whether post-training of large audio-language models (LALMs) actually improves audio understanding or merely adapts to question formats. They found that performance gains on benchmarks like MMAU and MMAR persisted even when audio was replaced with silence, but largely vanished when multiple-choice options were removed. This suggests that post-training teaches models to exploit textual cues rather than develop genuine auditory reasoning.

Agents

Agent memory benefits more from event aggregation than retrieval alone

The authors diagnose two distinct bottlenecks in agent memory, access and aggregation, and show that retrieval alone fails when answers require combining many events. They propose Trajectory Memory for lossless, question-time evidence access and TallyMem for deterministic aggregation of event specifications. TallyMem improves exact match on held-out MINTEval repositories from 35.2% to 49.0% over Trajectory Memory.

Language models

Vision scales multimodal learning without altering pure-text loss

The authors trained native multimodal models with either real image embeddings or content-free padding, holding text supervision and training configuration constant across model sizes and compute budgets. They found that visual content leaves the pure-text loss frontier and compute-optimal allocations nearly unchanged, but reduces multimodal loss with a relative benefit that grows approximately as a power law of compute. Text-only spatial reasoning also improves with scale, with the most sustained visual gains in localization, navigation, and symbolic planning.

Theory

Transformer text generation may never converge, but temperature ensures it

The authors study the asymptotic behavior of autoregressive transformers generating text indefinitely. They show that even a minimal three-token single-layer attention-only transformer can fail to converge. For a rich class of transformers including normalization, MLPs, and RoPE, they prove that above a model-specific softmax temperature, the empirical token distribution converges almost surely to a unique limit, independent of initial prompt. However, there is no universal temperature that forces convergence across all transformers—the required temperature can be arbitrarily high.

Evaluation & benchmarks

Diagnostic feedback drives iterative benchmark evolution revealing hidden LLM agent weaknesses

The authors propose EvoProbe, a framework that uses diagnostic feedback from LLM agent evaluations to iteratively generate new, more challenging tasks. In cybersecurity experiments, all eight models tested on 66 tasks showed decreased performance on updated tasks, with mean Outcome scores dropping 11.8% and mean Pass^3 rates falling 40.0%. The framework reveals phase-level failure patterns, such as wrong hypothesis and insufficient exploration, that aggregate scores obscure.

Code & software

Execution-verified feedback improves LLM harness optimization for SQL, Cypher, and Python generation

The authors introduce HERO, a two-stage framework for optimizing the harness (prompts and scaffolding) around LLM program generation. The offline stage learns reusable prompt updates from execution-verified successes and failures, while test-time refinement treats each generated program as an instance-specific harness. Across four SQL, Cypher, and Python benchmarks, HERO improves pass@1 by 7.6 points over the initial prompt, with a further 2.0 points from test-time refinement.

Evaluation & benchmarks

AI agents fail to produce honest scientific follow-up papers in new benchmark

The authors introduce Into the Unknown, a benchmark that tasks AI agents with writing a follow-up scientific paper given a research direction from an unpublished study. Across 135 submissions from nine frontier agents, none met all three criteria: integrity (no fabrication), completeness, and sufficiency as judged by domain scientists. 114 submissions failed the integrity check, most often due to claims of work that was never done or numerical contradictions with recorded evidence.

Agents

Tool discovery paradigm crossover: Retrieve-then-Plan wins on simple queries, Plan-then-Retrieve on complex ones at scale

The authors present a head-to-head comparison of two paradigms for tool-augmented LLM agents: Retrieve-then-Plan (RtP), which embeds the user query and retrieves similar APIs, and Plan-then-Retrieve (PtR), which first generates abstract tool stubs (hypothesized function signatures) to use as retrieval queries. Testing on ToolBench with 46,984 APIs across single-tool, intra-category multi-tool, and cross-category multi-tool queries, they find a complexity-dependent crossover: RtP outperforms PtR on single-tool queries, while PtR outperforms RtP on multi-tool queries. This crossover is robust across retrieval budgets, registry sizes, and model families.

AI for science & health

Orchestrating coding agents broadens autonomous computational research beyond optimization

Lange and colleagues present The Agentic Scientist, a pipeline that divides computational research into hypothesis generation, planning, experimentation, verification, analysis and manuscript writing. Across detailed case studies and 70 research campaigns, the authors report that coordinated coding agents produced testable findings and research artifacts, while orchestration and evidence preservation helped prevent the process from collapsing into simple performance optimization.

Generative media

Object-centric 3D memory enables consistent video generation under large viewpoint changes

The authors introduce Totem, a framework that represents each selected object as a persistent 3D memory in a shared world frame, combining object memories with scene geometry for target-view synthesis. They show that Totem achieves state-of-the-art performance on world-consistency and object-consistency benchmarks, particularly under challenging camera trajectories with occlusions, reappearance, and scale variation.

Safety & security

Cross-model KV cache reuse introduces safety incompatibility in language models

The authors systematically evaluate safety incompatibility in cross-model KV cache reuse across model pairs and reuse configurations. They find that value reuse consistently causes larger deviations from receiver-native safety behavior than key reuse. They propose VReparo, a lightweight technique that learns affine maps to align reused values, reducing average HarmBench attack success rate from 57.28% to 39.21% while retaining 85.62% of the time-to-first-token reduction.

Vision

Mixed-supervision method improves mammography segmentation for rare lesions

Dai et al. propose a two-branch framework for mammographic lesion segmentation that jointly trains on pixel-level masks for a few base lesion categories and image-level labels for a broader set of categories. A Lesion-wise Attentive Classifier uses category-specific feature prototypes to focus on lesion-relevant regions, reducing background interference. On multiple mammography datasets, the method consistently outperforms open-vocabulary baselines on both base and novel categories, narrowing the gap to fully supervised models.

Evaluation & benchmarks

Protocol risks in multi-agent systems depend on downstream agent decisions, not just local exploitability

The authors introduce Agent-Canary-Mas, an end-to-end system for measuring protocol risk in long-horizon multi-agent tasks. Using 975 matched malicious-benign task pairs across five business domains and three agent-to-agent protocols, they separate local attack success from final task harm. Local success often exceeds 90%, while consequence conversion ranges from 7.1% to 100.0%, and cross-agent evidence lifts attribution F1 from 29.7% to 95.5%.

Agents

Video agents improve their own harnesses using unlabeled video

Xu and colleagues present a framework that turns unlabeled videos into supervision for improving video-understanding agents. A Questioner creates questions and reference answers from structured video memories, while a Solver uses execution feedback to revise its skills, tools, parameters and coded workflows. The authors report progressive accuracy gains across evolution rounds and transfer improvements across model backbones.

Efficiency & systems

TempoKV compresses video generation KV caches by exploiting key-value predictability asymmetry

The authors propose TempoKV, a training-free quantization framework that compresses the KV cache, the growing store of attention keys and values, in autoregressive video diffusion models. They exploit an asymmetry: keys are predictable from their temporal history, while values are better predicted from the aligned current key plus a short history. Storing only low-precision residuals reaches up to 6.71x compression while improving generation quality over existing quantization methods.

Agents

Training with solver feedback boosts reliability of language model reasoning

The authors introduce Sequent-Prover, a family of language model agents that reason by writing and executing SMT-LIB programs with the Z3 solver. Using a novel training recipe called verifier-conditioned trajectory distillation, they post-train models by sampling multi-turn trajectories, filtering for correctness and non-triviality, and applying supervised fine-tuning and preference optimization. The 4B model's per-attempt accuracy rises from 85.6% to 97.0%, and four-sample reliability from 64.1% to 91.7%, while reducing average SMT calls and tokens.

Agents

Self-evolving visual agents learn tool use from unlabeled images using margin reward.

The authors propose Marvo, a two-agent framework that learns tool invocation for visual agents from unlabeled images. A challenger generates question-answer pairs, and a solver learns to answer them via reinforcement learning. A margin reward, defined as the solver's accuracy difference with and without tools, steers the challenger toward questions where tool use materially helps. Across 11,691 questions on agentic, visual search, mathematical, and general reasoning benchmarks, Marvo raises Qwen2.5-VL-7B's average accuracy by up to 7.1%.

Vision

Many contrastive training strategies for multimodal embeddings add little value

Li et al. systematically evaluate 12 contrastive training strategies for universal multimodal embedding models under a unified framework, finding that most offer no meaningful downstream gains and several degrade performance. Using a novel MLLM-as-a-Judge diagnostic, they identify three failure modes and show that jointly integrating hard-negative mining, false-negative mitigation, and hard-batch construction yields a 6.0% average improvement on the MMEB benchmark.

Agents

Task synthesis via artifact transitions yields long-horizon agentic trajectories.

The authors propose WorkBorne, a framework that models agentic tasks using artifact transitions—specifying who performs the work, which artifacts anchor the initial state and intended outcome, and the relation between them. They construct 1,061 artifact corpora and 7,882 executable task environments, producing 15,485 high-quality trajectories. Fine-tuning Qwen3.5-27B on these trajectories improves Terminal-Bench 2.1 accuracy from 43.1% to 61.8% and ClawEval General score from 75.0 to 80.2.

Evaluation & benchmarks

Skill-based benchmark reveals stark agent reliability gaps hidden by lenient scoring

The authors introduce SkillForgeBench, a benchmark that uses human-authored agent skills as procedural blueprints to automatically generate 842 sandboxed tasks spanning five professional domains. Evaluating eight mainstream agents reveals that strict, all-constraint satisfaction yields rates as low as 12.6% for weaker models and only 40.9% for the best (Opus 4.8), contrasting sharply with loose scoring rates of 70–85.7%. The work demonstrates that process rubric adherence is far rarer than completion rubric success, and that explicit skill guidance disproportionately benefits frontier models.

Agents

RL framework mitigates reasoning-tool trade-off in MCP agents

The authors introduce MCPDisCo, a reinforcement learning framework for agents operating under the Model Context Protocol (MCP). By explicitly modeling reasoning alongside routing, execution, and termination in a Thought-Markov Decision Process, and by mitigating the reasoning–tool seesaw through token-masked reward attribution and bidirectional gradient surgery, MCPDisCo achieves higher task success with fewer chain-of-thought tokens and generalizes to unseen servers and broader agentic tasks.

Theory

Near-linear time random Fourier features with strong concentration guarantees

The authors revisit the Fastfood random feature map for shift-invariant kernels, known to be computable in near-linear time but previously thought to give weak concentration. Using Brascamp-Lieb inequalities and the Schoenberg representation for radial kernels, they prove that Fastfood features yield near-subgaussian concentration for the Gaussian kernel and other radial kernels, closing a long-standing gap between structured and standard random Fourier features.