Research Threads

Adds a benchmark of 56 end-to-end physics research tasks created by 42 physicists to evaluate agent research workflows.

Intrduce ViBench 2.0, a benchmark decomposing coding agent tasks into feature requests and builds to reveal build-level failures not captured by unit tests.

Synthesizes long-horizon multi-tool tasks from real execution traces, enabling harder benchmark construction for agent process evaluation.

Provides a large-scale gameplay dataset with hierarchical subgoal annotations for fine-grained agent task evaluation.

Empirically identifies nine defect categories in deep research AI trajectories, enabling fine-grained error analysis.

Proposes monitoring observable AI behaviors as risk factors instead of directly measuring psychological harm, a new measurement approach.

Demonstrates that evaluation protocol choices (data split) can swing reported scores by 40 points, challenging metric reliability.

Shows that AI judges reverse correct verdicts after a single false note, demonstrating confirmation bias and affecting evaluation metric validity.

Intrduces SVGFaith, a meta-evaluation framework revealing that existing faithfulness metrics for SVG generation fail on spatial relations, and proposes a better metric.

Reveals silent bugs in agent leaderboard evaluation code that distort rankings.

Tests how agents use saved experience across tasks, informing design of memory architecture for retaining and retrieving relevant knowledge.

Demonstrates that agent wiki memory built from human diagnosis outperforms self-review, informing memory architecture design.

Compares explicit updated permission records against long history logs, showing that maintaining a concise current-state memory is more reliable for permission tracking.

Exposes persistent memory as a vulnerability surface and provides a benchmark for evaluating memory safety across sessions.

Adds cross-conversation memory and a shared sales playbook revised from chat outcomes without retraining.

Shows a generalist robot policy can improve broadly on untrained tasks by practicing diverse ones during adaptation.

Reveals that RL gains for language-model agents concentrate on error recovery and subtask execution reliability, not on choosing better actions.

Intrduces ReScopeBench, a benchmark evaluating how agents acquire, retain, revise, and use experience from evolving environments.

Combines failure-specific textual guidance with reinforcement learning to teach coding agents neglected behaviors without harming baseline performance.

Compares human diagnosis vs repeated self-review for agent learning from execution records, advancing self-improvement methods.

Adds a benchmark that tests retrieval, rejection, binding, and composition of visual evidence from many reference images in image generation.

Introdces Bio-VQA bench mark for biological visual QA, evaluating whether models ground visual evidence for counting and individual identification.

Proposes Diagnosis by Composition, which aligns radiological findings with text embeddings, improving evidence grounding for unseen disease diagnosis.

Reveals that attention weights are not fully indicative of how VLMs select relevant images in multi-image contexts.

Introduces a long-form video QA benchmark revealing the limitations of current models in reasoning over extended multimodal stories.

Decomposes what 'verified' means for LLM generated code, showing verifier acceptance can be high but spec binding is low, promising static analysis with LLM hybrid approaches.

Couples an LLM-agent with the Lean 4 proof assistant to formally verify GPU kernels against reference implementations, a direct hybrid of static verification and language model code generation.

Introduces SALT, a static type inference plus LLM pipeline that raises annotation accuracy while cutting LLM cost by 75%.

Finds that discarding reasoning steps from training data does not harm agents on standard benchmarks, questioning the need for thought supervision in distillation.

Identifies a systematic under-weighting of high-confidence tokens in on-policy distillation, which distorts the student's learned distribution away from the teacher's guidance.

Demonstrates that a compact student (4B parameters) can reach 93% of a teacher's performance on real robot tasks using a fraction of accelerator time, advancing distillation efficiency for embodied agents.

combines trajectory-level intervention and step-level correction to improve RLVR on hard problems.

Introduces teacher-evolution self-distillation where teacher receives feedback to improve distillation alignment

Proposes locally differentially private watermarks that bound single-text inference while enabling population-level estimation of AI text prevalence.

Introduces an automated pipeline to discover training data side effects on model behavior, expanding auditing to side-effect discovery.

Formsizes temporal membership inference for models updated with data insertions and deletione, extending privacy auditing to continual learning.

Investigates the specificity of interventions to suppress personally identifiable information, challenging targeted mechanisms.

Reveals a counterintuitive link between higher factual memory (facts seen once) and greater scientific discovery, informing how training data rarity affects model behavior.

Reals 19 channels where document extractors read invisile content, enabling attacks on LLMs without alering user visile content.

First systematic benchmark evaluating inter-agent influence capabilities (persuasion, deception, coercion) in multi-agent LLM systems.

Shows how fabricated hazard reviews can steer AI agents to reject legitimate options, a security vulnerability.

Introduces an evolving attack strategy that adapts to defender responses, posing new challenges for search agent security.

Outlines that agents may detect scam warnings but still proceed, highlighting a safety vulnerability in agent deployment.

Presents LiveFoley, a streaming video-to-audio model that generates real-time sound from pixel input, adancing real-time multimodal generation.

Compares vLLM and SGLang, showing chunked prefill drives speed differences, which informs cache optimization.

Chunk-level selection improves long-context inference throughput by reducing KV cache reads.

Reveals that mid-depth layers are better than the owning layer for scoring KV-cache evictions, improving cache management decisions for inference throughput.

Adds a training-free KV selection scheme that precomputes which historical tokens to keep, cutting long-context memory traffic.

Introduces a distributed path reservation mechanism to avoid routing collisions for KV cache transfers in inference clusters.

Calibrates reward models for emotional fidelity in video generation by learning which cues are reliable across frames and RL iterations.

Uses self-judgments with a gradually growing rubric to train language models for open-ended writing improvement.

Forecasts how language model responses change under reward edits, aiding reward design.

Trains an action-probability head over action menus rather than generating text, reformulating the RL reward signal to focus on correct choice rather than token-level generation.

Proposes a return distribution approach (not a single scalar) for value estimation in RL, improving performance when training data is scarce.

Proposes NanoKV for compressing stored history in autoregressive video models, improving reconstruction quality with less memory.

Trains a neural router to predict which historical frames are most useful for future generation instead of using static retention.

Introduces Wave Forcing, a co-designed generation and caching framework that enables continuous, multi-step video generation at near-watchable speed.

Introduces Prism, a method where models improve both understanding and generation by learning from their own outputs.

Presents D3D, which overlaps drafting with verification to accelerate text generation.

Finds that verifying fewer draft tokens per round makes hybrid models faster.

Parallelizes on-policy distillation for speculative decoding, accelerating draft model training.

Shows that allocating more attempts to harder inputs is near-optimal only on easy workloads, revealing limitations of current test-time scaling practices.

Proposes a two-level data selection rule for offline RL based on policy-target mismatch, cutting training tokens by two thirds while improving accuracy.

Shows reward-trained image models can degrade with more generated candidates, questioning test-time scaling of RL post-trained generators.

Introduces a profile-guided dynamic programming method for latency-minimizing partitioning across heterogeneous GPUs and emulators.

Uses research tasks derived from physicists' own work to test agents in a realistic deployment context.

Introduces FinDendro, a benchmark of dated financial facts, testing real-world fact retrieval in a deployment-relevant domain.

Generates thousands of abstract reasoning puzzles based on spatial and semantic concepts and tests reasoning models, revealing failures on non-linguistic operations.

Introduces LifeVal, a benchmark built on consequential personal tasks like managing finances and finding childcare, reflecting real-world deployment.

Shows how evaluation code errors affect real-world leaderboard rankings, challenging ecological validity.

Finds that merging specialists does not require matching training adapter capacity, tying model merging to pruning concepts.

Studies the interaction of pruning with quantization and fine-tuning, revealing degradation when techniques are combined on small models.

Shows that pruned models can retain problem-solving coverage on math and code tasks even when one-shot accuracy drops, using pass@k to separate coverage from sampling efficiency.

Demonstrates gradual pruning with lightweight adjustment training can remove 90% of weights more cheaply than prior pruning.

Proposes finetuning that temporarily deletes safety neurons to teach alignment without them, and shows resistance to pruning-based jailbreaks, illuminating safety neuron dependence.

Shows that encoder-based context compression systematically weakens refusal alignment, a previously undocumented failure mode in instruction-tuned models.

Shows that models can pass deletion checks yet diverge later, revealing temporal dynamics in unlearning effects.

Shows that suppressing misaligned behavior from a partially misaligned state leads to longer-lasting fixes, contributing to understanding alignment dynamics.

Demonstrates that fine-tuning on false facts a model disbelieves causes broad misalignment, revealing risks in alignment training.

Introduces a hybrid reasoning framework that gating decides whether to emit an explicit token or feed hidden state directly, reducing trace length without losing accuracy.

Trains models to locally decide when to stop reasoning based on internal knowledge state, directly controlling trace length.

Proposes Low-Rank Hierarchical Merging to shorten reasoning traces without retraining.

Filters wrong turns in long chain-of-thought traces to shorten answers and improve accuracy.

Documents that LLMs asked to translate to programs first solve the task in 85% of traces, a reasoning trace content control failure.

Exposes a new vulnerability in offline RAG: integrity of precomputed embeddings is not verified, enabling adversarial hijacking even when tolerance checks pass.

Illustrates how poisoned documents that repeat user queries can bias document selection in retrieval-augmented systems.

Demonstrates scoring imagined future rollouts to ground action selection in simulated outcomes, akin to verifying candidate plans before execution.

Shows that a language model storing web page identifiers can outperform embedding search for grounded web navigation.

Trains a search agent to rank documents by relevance to original user question

Compresses tool outputs into latent tokens, reducing token consumption by 70-89% and runtime by 24-56% while improving agent performance, directly optimizing agent harness cost.

Introduces WebCatalyst, a RL framework using browser games to train LLMs for interactive HTML generation, achieving performance surpassing frontier models with few training requests.

Decouples description and assessment in VLMs to eliminate confirmation bias in robot task evaluation.

Augments vision-language agents with classical task and motion planning to boost household task success, an optimization of the agent's decision pipeline.

ParetoSaddler automatically optimizes agent scaffolding (prompts, tools, middleware) jointly for quality and cost.

Distills a latent motion prior from human soccer capture into a humanoid policy for multiple skills, demonstrating action transfer across human to robot embodiments.

Learns a shared action representation from human and robot videos, enabling action transfer.

Trains bimanual robot tasks from handheld demonstrations, enabling action transfer from human to robot.

Introduces a framework that adapts world models for real-time interactive exploration by agents and humans, targeting embodied training in virtual environments.

Shows that predicting 3D motion from human video, rather than pixels, improves robot policy learning, grounding world models in motion prediction.

Provides a large-scale multi-platform simulator world model from diverse robot demonstrations for policy learning.

Introduces a pre-execution scorer that evaluates multiple imagined futures from a world action model before selecting an action, improving robot success rates.

Learns and revises a billiards dynamics model from self-chosen robot interactions for manipulation.

Studies detection of semantically invalid robot commands (e.g., swapped axes) using a model that links command structure to observed outcome, extending grounding to action-level correctness.

Shows terminal tool calls requiring confirmation are treated as completed without the operation finishing, a tool grounding failure.

Apriel Voice 27B processes speech directly for reasoning and tool calling, grounding speech semantics without separate transcription.

evaluates tool-call sequence prefixes for executability.

Redefines GUI click correctness by outcome rather than bounding box, providing a more accurate grounding metric for tool actions.

Finds that reasoning models share systematic errors on conceptual vs. operational transformation puzzles, providing evidence for correlated failures.

Demonstrates that averaging independent models outperforms a single larger model on fixed data, related to correlated errors.

Finds that reasoning increases inter-model agreement while also increasing shared mistakes, adding evidence on correlated errors across reasoning models.

Presents HALO, a humanoid system that learns camera-to-movement policies from simulated demonstrations and transfers to real homes, advancing balance and navigation.

Uses hierarchical RL with human motion priors to learn balanced locomotion, dribbling, and shooting in a humanoid soccer robot.

Proposes an orchestration system that re-targets task-level goals when balance, contact, or disturbances change during humanoid execution.

Adds a systolic linear attention design that reshapes attention around accelerator hardware to speed up training.

Adds a VLA architecture that feeds raw image patches into a shared vision-language backbone and shows this improves robustness to camera shifts.