Sat 31 Oct
LiveGet it free
ZOTPAPER

News without the noise

Research

New AI papers from arXiv, picked daily and explained with their claims, benchmark results and code.

2566 articles · page 31 of 54 · The Research front page

AllAgentsLanguage modelsReinforcement learningVisionGenerative mediaSpeech & audioRobotics & embodied AISafety & securityEvaluation & benchmarksEfficiency & systemsCode & softwareAI for science & healthTheory
Viewgridlistcompact
AI for science & health

Soft routing among process experts helps PDE forecasting, but gains prove seed-dependent

The authors propose COSMOS, a neural operator that densely mixes four process-biased specialists with a learned continuous gate instead of sparse top-K expert selection. On PDEBench families, a locked three-seed run beat a tuned FNO on diffusion-reaction and matched it on Navier-Stokes, but an 11-seed audit showed the diffusion-reaction gain was unstable. Soft routing beat hard top-1 routing by 2.2x in error, yet replacing the learned gate with the generator's true mixture weights further lowered error, and routing labels did not align with physical mechanisms.

·3 min
Code & software

Program-graph evidence and blinded LLM surrogates detect code equivalence gaps

The authors present FEAgent, a system that assesses functional equivalence of code by combining typed program-graph analysis with differential surrogate execution. Instead of comparing programs directly, FEAgent uses a branch-aware generator to propose discriminative inputs and two blinded LLM surrogates to independently predict observable behavior. On EquiBench and SWE-bench Verified, execution-grounded adjudication confirmed that 18% of benchmark labels and 28% of unit-test-passing patches conceal behavioral divergences.

·3 min
Safety & security

Learning from failure enables LLM agents to resist prompt injection attacks

The authors propose Self-Reflection Fine-Tuning (SRFT), a training framework that exposes LLM agents to their own failure trajectories under prompt injection attacks and uses an expert model to generate self-reflection rationales. On AgentDojo, SRFT reduced attack success rate (ASR) from 7.59% to 1.26% on Llama-3.1-8B-Instruct and from 16.97% to 1.05% on Qwen3-8B, while maintaining task utility; under adaptive attacks (RL-Hammer), SR-Agent achieved ASR below 20%, outperforming prior methods such as Meta SecAlign which exceeded 60%.

·2 min
Evaluation & benchmarks

Simple choices outperform complex recipes in iterative reward design benchmarks

Bhamidipaty and colleagues introduce BIRD, a framework that places iterative reward design methods under matched feedback conditions, implementations and policy-training budgets. After comparing six methods and ablating 30 design choices, the authors combine the most reliable components into two algorithms that significantly outperform the evaluated prior methods in their respective settings.

·3 min
Evaluation & benchmarks

AutoSciBench automatically generates scientific agent benchmarks that adapt as agents improve.

Kaplan et al. introduce AutoSciBench, a framework that automatically generates and iteratively refines benchmark tasks for scientific AI agents. By representing tasks as high-level concepts and low-level recipes, then using solver trajectories to close shortcuts and shift tasks toward deeper reasoning, AutoSciBench produced benchmarks that reduced solver accuracy by 22.4 and 25.5 percentage points relative to human-curated benchmarks in computational biology and materials science respectively, while receiving higher average quality ratings across all three tested domains.

·2 min
Reinforcement learning

Outcome-guided self-distillation improves LLM reasoning by adapting supervision

On-policy self-distillation (OPSD) provides dense token-level supervision but suffers from noise and instability. The authors analyze forward and reverse KL divergences and find that teacher reliability degrades on longer incorrect trajectories. They introduce Outcome-Guided OPSD (OG-OPSD), which applies forward KL on correct trajectories and reverse KL on incorrect trajectories only up to a prefix determined by cumulative teacher entropy. OG-OPSD consistently outperforms vanilla OPSD and strong baselines on mathematical reasoning, multimodal reasoning, and out-of-distribution tasks across Qwen3 models from 1.7B to 8B parameters and Qwen3-VL-2B.

·3 min
Generative media

Geometry-aware alignment keeps diffusion models near natural image structure

Wang et al. argue that standard preference optimization can gradually push diffusion models away from the low-dimensional structure associated with natural images, reducing quality and diversity. Their proposed method, Anisotropic Geometry-Aware Preference Optimization (APO), adjusts regularization by direction and reportedly beats several alignment methods with average win rates above 60 percent across multiple benchmarks.

·3 min
Agents

Compiling agent trajectories into targeted long-context tasks boosts model capabilities

The authors introduce TrajLong, a framework that compiles LLM agent trajectories into long-context training tasks targeting three atomic capabilities: evidence grounding, cross-evidence aggregation, and temporal state maintenance. Mid-training Qwen3-14B and 30B models with TrajLong data followed by supervised fine-tuning yields broad gains on 6 long-context and 12 agent benchmarks, outperforming both raw and masked trajectory baselines under matched training budgets.

·2 min
Vision

Distribution drifting on spatial features enables efficient one-step super-resolution

The authors propose DriftSR, a one-step framework for real-world image super-resolution that uses a frozen diffusion model’s intermediate features. By treating spatial locations as samples for distribution drifting and using structure-modulated guidance, it achieves high perceptual quality without score distillation or adversarial training. The method outperforms previous one-step methods on real-world benchmarks while maintaining inference efficiency.

·2 min
Safety & security

Auditing pending actions catches staged attacks in long-running AI agents

Liu and colleagues demonstrate staged prompt injections that enter an agent through external content, propagate across multiple steps, and later cause harmful actions without preventing task completion. They formulate boundary action auditing, which evaluates each consequential message or tool call before execution, and report that their training-free Path-Aligned Attribution method detects substantially more attacks with fewer false blocks than the tested baselines.

·3 min
Agents

Workspace-grounded synthesis scales verifiable training for long-horizon work agents

Zhang and colleagues introduce WorkForge, a pipeline for creating training environments in which agents must navigate multiple files and produce professional deliverables over extended interactions. Post-training a Qwen3.5 model on solver trajectories from these environments produced large reported gains on two work-agent benchmarks, while controlled experiments found benefits from both more data and longer trajectories.

·2 min
Safety & security

Code execution exposes oversight failures hidden by aggregate benchmarks

The authors introduce PyINE, an experimental framework in which instrumented Python execution supplies authoritative labels for model predictions, intermediate claims and oversight decisions. In its first implementation, a deliberately trained shortcut-following model became better at predicting program outcomes but still failed systematically when comments and other human-facing cues contradicted runtime behavior. Evaluations showed that overall oversight scores can conceal poor detection of these rare but important errors.

·2 min
Safety & security

Causal interventions often succeed via hidden pathways, not natural mechanisms

The authors plant silent hidden pathways inside GPT-2 small, Pythia-1.4B, Pythia-6.9B and Qwen2.5-7B, then run 100,800 causal interventions to determine which succeed through these wrong mechanisms. They find that site-level distance metrics score below chance (AUROC 0.35-0.47) at flagging such successes, while their label-free detector, Hidden-Pathway Contribution (HPC), achieves AUROC >= 0.99 on visible pathways but fails on in-range ones.

·3 min
Efficiency & systems

Training models for deployment preserves quality under four-bit quantization

Xu and colleagues present QATFactory, an open-source framework that trains large language models to tolerate the numerical errors introduced by deployment-grade quantization. Across models from 8 billion to 230 billion parameters, quantization-aware distillation produced more accurate deployed models than post-training quantization, with the best training recipe depending on the target four-bit format.

·2 min
Agents

Mined agent skills need domain-specific evidence and format choices

The authors compare six combinations of mining evidence (successful traces only, successes and failures with labels, or the same mix without labels) and skill form (ordered workflow plans or declarative ontologies) while holding the mining pipeline fixed. They find that no single configuration dominates: the best choice depends on the domain and task structure, with workflows and contrastive evidence helping in some settings and ontologies in others.

·2 min
Safety & security

Per-key f-divergence constraints enable secure multi-bit text watermarking

The authors introduce a framework for multi-bit text watermarking that defines security through per-key f-divergence from the base language model distribution. They develop SimpleMark, a coding-based scheme that optimally biases next-token distributions subject to a prescribed divergence budget, and experimentally show it achieves substantially lower watermark detectability than prior methods while maintaining competitive message-recovery performance and generation quality.

·3 min
Efficiency & systems

Three-step hybrid pipeline validates GPU kernels by combining concurrency testing with formal proof

The authors present RESOLVE, a GPU-kernel validation framework that separates concurrency correctness from functional correctness. In a three-step pipeline, it tests for nondeterminism via timing perturbation, rewrites kernels into reduced-concurrency versions that preserve bitwise behavior, and then formally proves equivalence using the F*/Pulse framework. RESOLVE validated kernels from KernelBench and fused GEMMs across CUTLASS, Triton, and Gluon, and found four previously unreported issues in mega-kernels, including two clear bugs.

·3 min
Agents

TeleTune learns agent skills from unfiltered user logs without goals or replay

The authors introduce TeleTune, a method for building a library of textual skills and workflows from offline telemetry logs that have no recorded goals, cannot be replayed, and may interleave multiple tasks. Using only action-prediction errors on the logs as a learning signal, TeleTune proposes edits to the library and retains only those that improve accuracy on held-out trajectories. On WorkArena and Online-Mind2Web, TeleTune achieves average success rates of 77.1% and 80.6%, outperforming the strongest baselines by 6.7% and 7.7% respectively.

·2 min
Generative media

Variable-size tokens cut diffusion costs where less detail is needed

Nakayama et al. introduce Level-of-Token Diffusion, which assigns smaller tokens to detail-sensitive regions and larger tokens to simpler areas such as skies, flat surfaces or blurred backgrounds. By shortening the sequence processed by a diffusion transformer while retaining full-resolution denoising updates, the method achieved reported average speedups of up to 2.04 times for images and 3.53 times for videos.

·2 min
Vision

Probability rewards keep visual grounding reasoning aligned with predicted objects

Sun et al. study thinking drift in visual intention grounding, where a vision-language model identifies an object from a goal-oriented query and predicts its bounding box. They show that reinforcement learning based only on box overlap can improve localization while reinforcing reasoning that names a different object. Their proposed Rita framework reduces this mismatch without requiring labeled reasoning traces.

·2 min
Safety & security

Backdoor attacks on interactive video generation make outputs freeze when action triggers appear.

The authors propose BadAction, the first backdoor attack tailored to the action control channel of interactive video generation (IVG) models. By pairing poisoned action sequences with static target videos, the backdoored model generates frozen frames that ignore subsequent user actions when triggered. Experiments on a representative IVG model show 91.0% attack success with action-only triggers and 80.4% with tri-modal triggers, while preserving benign generation and evading existing backdoor defenses.

·2 min
Generative media

Steepest Guidance aligns generative models by maximizing local reward improvement

Takakura et al. treat inference-time alignment of flow and diffusion models as sequential optimization in probability measure space. They introduce Steepest Guidance, which steers the generative process along the steepest ascent direction of the reward functional. The method achieves unbiased estimation for linear rewards, handles non-linear rewards via particle approximation, and consistently outperforms existing training-free approaches.

·2 min
Code & software

Expert verification failures become reusable guidance for coding agents

The authors developed a verification kit that captures expert diagnosis of verification failures into reusable procedures, combined with executable language definitions in the K framework. On the HumanEval benchmark, a human-guided campaign achieved a perfect 164/164 success rate after two targeted repairs, as confirmed by final AI audit. Additionally, audits of 12 clean/defective packages successfully identified all defects while accepting all clean packages, demonstrating the kit's ability to detect problems that proofs alone miss.

·3 min
Reinforcement learning

Smaller frozen models provide better rejected responses for preference distillation

The authors conducted a large-scale study on preference distillation, comparing reject sources from smaller frozen models versus the student's own responses. They found that across student model sizes from 7B to 72B, rejects from smaller Base models consistently produced stronger student performance on code generation and mathematical reasoning, while requiring less compute. Theoretical analysis and interventions identify task structure and limited reference policy coupling as key properties of useful rejects.

·3 min
Generative media

Spatial resolution hierarchy enables parallel mesh generation with explicit topology

The authors propose MeshOctave, a discrete diffusion model that generates artist-quality meshes by defining scale through spatial grid resolution. Coarsening merges vertices in the same voxel (resolution collapse), and refinement is factorized into per-face decisions about octant children and connectivity (split-and-rewire). This yields parallel intra-scale generation and dynamic-length meshes, achieving better geometric fidelity and topological validity than strong baselines.

·2 min
Safety & security

Machines can unlearn deception itself, not just the knowledge it distorts

The authors propose PACT, a machine unlearning objective that removes context-triggered deception from LLM weights rather than deleting the underlying knowledge. On two 32B reasoning models, deception fell to under 3% on held-out scenarios, while system-prompt adherence, secret-keeping and reasoning traces remained at base model levels. Standard unlearning objectives either left deception in place or induced a new failure mode the authors call context blindness.

·3 min
Language models

Routing attention heads by token enables joint scaling of parameters and context

Fu et al. introduce NAMOH, an architecture-native sparse attention mechanism that treats each attention head as an expert. A learned router selects K out of H heads for each token, and each head performs causal attention only on the tokens routed to it. This design reduces per-token KV access to approximately (K/H)² relative to full H-head attention, while experiments show NAMOH outperforms dense models with the same total parameters and enables more efficient long-context inference than smaller dense models matched in active parameter count.

·3 min
Safety & security

Sparse attention in LLMs leaks private data via GPU side channels

Chen et al. identify a new side channel, Sparsity-Induced Memory Access (SIMA), arising from input-dependent key-value cache access in sparse attention. They present SparLeak, a phase-aware attack that extracts SIMA traces from both prefill and decoding phases, achieving average success rates of 90.9% for attribute inference and 87.3% for response reconstruction across three LLM architectures and three sparse attention mechanisms.

·3 min
Efficiency & systems

A library that makes gradient-based training data attribution practical for large language models

The authors introduce dattri-LLM, a library for training data attribution (TDA) at LLM scale. It uses a FLOP-aware cost model to efficiently represent per-example gradients, and non-invasive hooks to capture gradients from existing training loops. On benchmarks, it achieves on average 3.2x the throughput of the fastest competing library and scales to 110B-parameter models across four H200 GPUs.

·2 min
Robotics & embodied AI

Wiring details determine whether affordance heads help or hurt VLA policies

The authors systematically ablate how an affordance head is injected into a vision-language-action policy. They show that a stop-gradient is mandatory—without it, the policy collapses. With the backbone protected, the dominant factor is the bridge initialization: a randomly initialized residual bridge (allowing immediate head influence) yields 96.2% success on LIBERO, compared to 93.1% without the head.

·3 min