failure attribution
When an AI agent fails at a task, failure attribution is the structured detective work of figuring out exactly *why*. It traces back through the agent's chain of steps and observations to pinpoint the specific decisive error that started the cascade, moving beyond a simple failure signal to root-cause analysis.
The papers show that naive attribution, like blaming the last step, is a surprisingly strong but limited baseline. Robust methods instead build dependency graphs to formally trace how early mistakes propagate into final failures. Most importantly, they demonstrate that accurate attribution is practically powerful: pinning down the exact cause enables targeted repair or reflection, which dramatically lifts success rates on the next attempt.
- Papers
- 140
- Released code
- 11
- First seen
- June 2026
- Latest
- Nov 2026
122 papers in the last two months, against 13 in the two before.
Start here
Three papers to read first, in the order they came out- 1Structured backtracking beats reflection for long-horizon language agents
VBL, a training-free extension of ReAct, maintains a goal stack with dual-horizon verification and typed failure diagnosis, improving task success across three benchmarks.
Nov 2026 - 2Simulator bugs cause most robot policy failures in common benchmarks
A fixed threshold in PhysX's collision detection creates false failures in robot simulation benchmarks; correcting it eliminates them.
Nov 2026 - 3Language model libraries often fail outside their core calculation code
GPU kernel library failures often originate in build and runtime environments, while specialized assumptions and inadequate testing allow correctness and hardware-specific problems to slip through.
Nov 2026
How fast it is moving
Often studied with
agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real44 togetherverificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.36 togetherbenchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.22 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.CV
Image search models can retain left-right information but ignore it
A patch-grid readout restores left/right image search by recovering spatial relations that vision-language similarity either hides in pooled image summaries or discards during pooling.
Huawei Technologies Canada · Nov 2026
- Top Universitycs.ROcode
Simulator bugs cause most robot policy failures in common benchmarks
A fixed threshold in PhysX's collision detection creates false failures in robot simulation benchmarks; correcting it eliminates them.
Tsinghua University, University of Chinese Academy of Sciences · Nov 2026
- Big Techcs.AI
Language model libraries often fail outside their core calculation code
GPU kernel library failures often originate in build and runtime environments, while specialized assumptions and inadequate testing allow correctness and hardware-specific problems to slip through.
Microsoft Research Asia, Tsinghua University · Nov 2026
- Chinese Techcs.AI
Structured backtracking beats reflection for long-horizon language agents
VBL, a training-free extension of ReAct, maintains a goal stack with dual-horizon verification and typed failure diagnosis, improving task success across three benchmarks.
City University of Hong Kong, Huawei Noah's Ark Lab · Nov 2026
- Top Universitycs.AI
Fast AI agents read states correctly yet still choose wrong actions
Fast agent models mostly read state correctly but still act wrongly, due to compositional and sub-goal failures.
Carnegie Mellon University · Oct 2026
- Big Techcs.SE
Coding agents often miss crashes and misread changes that prevent them
Coding agents miss many real system crashes, and correct predictions on crashing code alone can conceal failures to distinguish safe execution from failure.
Columbia University, Google Inc. · Oct 2026
- Academiccs.AI
Structural baselines mask real progress in agent failure detection
Structural baselines account for most gains reported in agent failure detection and attribution.
Chemnitz University of Technology, Fraunhofer Institute for Molecular Biology and Applied Ecology · Oct 2026
- Independentcs.CR
Poisoned training tasks can backdoor self-improving AI agents
Poisoned evolution tasks can trick self-improving AI agents into building backdoors into their own skills.
Nov 2026
- Big Techcs.MA
AI agents often repeat failed strategies instead of using corrective feedback
GPT-4o agent pairs succeed in 60.5% of asymmetric puzzle conversations; a taxonomy-guided intervention raises success to 70.5%.
Microsoft Research · Nov 2026
- Academiccs.SE
Coding agents can report successful hardware migrations while changing model training
Coding agents migrating PyTorch from GPU to TPU can pass execution checks while silently changing training, and many failures raise no error, so independent verification is essential.
North Carolina State University, Federal University of Ceará · Oct 2026
- Top Universitycs.AI
Customer-service AI agents can improve from their own mistakes after deployment
An in-house mid-sized customer-service model that retrains on its own verified failures, using diagnoses as hints for itself, cut its error rate below the frontier model it replaced.
Meituan, University of Melbourne · Nov 2026
- Chinese Techcs.MA
Multi-agent AI systems improve by learning from execution traces
MAS-Loop closes the feedback loop in multi-agent systems by using execution trajectories to refine and improve systems across training iterations.
Hong Kong University of Science and Technology, Ant Group · Oct 2026
- Big Techcs.LG
Global coordination of proof agents beats local repair in formal theorem proving
Maestro coordinates prover workers globally to handle failures, outperforming existing provers on mathematics and verification benchmarks.
University of Illinois Urbana-Champaign, AWS AI Labs · Oct 2026
- Top Universitycs.CV
VLMs mostly err by ignoring visual evidence they possess
Most vision-language model errors are due to failing to use visual information that is already encoded.
Technion · Oct 2026
- Top Universitycs.CLcode
LLMs fail multi-step math despite solving each step correctly
LLMs often solve all intermediate math milestones yet fail to combine them into a correct final solution, making composition their largest diagnosed reasoning gap.
Rutgers University, Harvard University · Oct 2026
- Big Techcs.CL
Foundational LLM capabilities only partially predict agentic execution gaps
Foundational benchmarks partially predict LLM agent operation performance; operation profiles improve task-score prediction.
Nankai University, Hong Kong University of Science and Technology (Guangzhou) · Oct 2026
- Industrycs.AI
Dataset turns failed agent traces into diagnosis and repair supervision
A dataset of 50,228 error-diagnosis pairs enables LLM agents to learn from failures, improving correction success and diagnosis accuracy.
Apodex · Oct 2026
- Top University
Retrieval-based checks miss many factual errors in open-ended medical answers
Retrieval-based factuality evaluation of long-form medical answers fails because scaling retrievers, verifiers, reasoning, and medical tuning does not fix fundamental error modes.
Center for Language and Speech Processing, Johns Hopkins University · Sept 2026
- Chinese Techcs.AI
Dependency graphs pinpoint decisive errors in long agent execution traces
DeFA achieves the highest accuracy in attributing decisive errors and responsible agents in LLM agent trajectories.
Qwen DianJin Team, Alibaba Cloud Computing, Beihang University · Oct 2026
- Chinese Techcs.AI
Diagnosis-first framework boosts solver performance across numerical domains
ADSD links numerical diagnosis to reusable solver skill discovery, improving solver accuracy, robustness, and efficiency across four numerical domains.
University of California, Berkeley, Alibaba Group · Oct 2026
- Big Techcs.CL
Controlled prerequisites reveal why multimodal models fail composite tasks
Causal task decomposition isolates intrinsic capability failures from cascading prerequisite errors in MLLMs.
Google Research, Stanford University · Oct 2026
- Chinese Techcs.CVcode
Motion-space analysis exposes object movement flaws in generated videos
MotionInsight diagnoses object motion deficiencies in generated videos by scoring object consistency, motion continuity, and physical plausibility from motion-space representations.
MMLab, CUHK, ByteDance Inc. · Oct 2026
- Chinese Techcs.AI
A method that links failure localization to repair by identifying which editable system asset to change
DAAF learns the effects of replacing system assets to decide which edit fixes LLM agent failures.
Tencent · Sept 2026
- Big Techcs.CV
AI models can read diagrams correctly and still miscount their contents
Gemini models usually fail to count diagrams because of computation rather than perception, but reading errors become the bottleneck on larger drawings.
Google · Nov 2026
- Top Universitycs.RO
Robot memory that runs by clock confuses waiting with working
Time-sampled memory makes robots mistake idle waiting for task progress, causing early stopping during null events.
Nanyang Technological University · Nov 2026
- Big Techcs.RO
Robot coding agents learn faster with smarter feedback, not more data
Agentic robot self-learning is bottlenecked by agents' ability to interpret physical interactions into effective policy updates, not by access to information.
UC Berkeley, Bosch · Nov 2026
- Academiccs.CL
AI agents complete roughly four times as many tasks with helpers
Reusable primitives with embedded LLMs, composed per task by HEELO, solve real-world tasks that frontier models fail to complete.
University of Illinois Urbana-Champaign, Georgia Institute of Technology · Nov 2026
- Independentcs.AI
Tool-call pipelines often alter what language-model agents actually execute
LLM agent harnesses frequently alter tool calls in transit, causing silent misexecution, inflated costs, and failures wrongly blamed on the model.
Oct 2026
- Research Labcs.SEcode
Coding agents rarely build the agent a user actually asked for.
Coding agents fail to build agents that satisfy held-out requirements, overfitting to visible cases and falsely reporting completion.
Beijing Academy of Artificial Intelligence, Institute of Automation, Chinese Academy of Sciences · Oct 2026
- Top Universitycs.AI
Attribution-guided skill graphs improve targeted repairs for frozen language models
GraphSkillAA uses attribution-guided updates and two-level validation to optimize skill graphs, outperforming existing methods.
National Key Laboratory for Novel Software Technology, Nanjing University · Sept 2026
- Chinese Techcs.AI
Targeted reflection improves multi-agent systems by locating decisive errors
Multi-agent systems improve 22-27% by identifying and reflecting only the agent that caused the failure.
Renmin University of China, Ant Group · Sept 2026
- Big Techcs.AI
Structured behavioral abstractions improve diagnosis of failures in LLM agents
AGENTSCOPE diagnoses LLM agent failures by analyzing structured behavior graphs against neural invariants.
Tsinghua University, Microsoft Research · Sept 2026
- Top Universitycs.CL
Language model RL failures can guide targeted pretraining repairs
Failure-linked n-grams in RL rollouts can retrieve targeted pretraining data that improves reasoning more than equal-sized randomly sampled data.
Princeton Language and Intelligence · Oct 2026
- Top Universitycs.AI
Linking memory back to raw data helps AI agents avoid repeating bad lessons
Evidence-linked memory, storing lessons with their source trajectories, reduces failures caused by incorrect or misapplied lessons in LLM agents.
KAIST, DeepAuto.ai · Nov 2026
- Big Techcs.AI
AI agents that spot workplace tasks still struggle to finish them
Task discovery and task execution are distinct capabilities for proactive workplace agents.
Microsoft · Nov 2026
- Industrycs.SE
Coding agents fail most long chains of dependent tasks
Coding agents solve at most 18.5% of long-horizon multi-step software engineering tasks.
Klavis AI · Nov 2026
- Chinese Techcs.CL
Reusable parts let a single fix repair many broken AI training environments
Building agent training environments from reusable, evolvable pieces, then repairing the pieces when an environment breaks, raises how many work and improves agent training.
Shanghai Jiao Tong University, Tencent · Nov 2026
- Chinese Techcs.AI
Defect-guided fuzzing finds deep research AI failures three times faster
Deep Research systems have nine recurring defects that EMBERS exploits to detect failures at 3.21 times the baseline rate.
Tianjin University, Huawei Technologies Co., Ltd. · Nov 2026
- Chinese Techcs.RO
Robot action checks trade off error detection against pinpointing causes
World models trained on clean transitions detect and localize silent action-interface mismatches that are numerically legal.
Yale University, Ant Group · Nov 2026
- Big Techcs.CL
Checklist rubric content dominates LLM judge strictness, not the model
Checklist content, not judge choice, determines LLM-grading strictness, while ungated model self-review degrades rubrics by removing useful checks and adding unstated requirements.
Microsoft · Oct 2026
- Top Universitycs.AI
Backtesting method lets AI agents forecast past events for instant feedback
FABench reveals retrieval helps agents match market forecasts, yet evidence weighing remains a critical failing.
Raven Labs, Zhejiang University · Nov 2026
- Big Techcs.AI
AI agents learn more from human diagnosis than repeated self-review
Agents that organize their own procedural memory as linked wiki pages, with humans diagnosing failures rather than designing schemas, pass more enterprise tasks than baselines while storing less.
University of Illinois Urbana-Champaign, Amazon · Nov 2026
- Big Techcs.MA
AI agents improve on specialized tasks with 17–38% fewer training runs
EMMA treats multi-agent configurations as episodic memory with explicit, guarded updates, outperforming baselines by up to 32.5%.
Amazon, University of California San Diego · Oct 2026
- Chinese Techcs.AI
AI agents can improve by tracing how shared evidence changes answers
SPLICE locates information gaps in multi-agent workflows, adds one edge to deliver missing evidence, and adopts only when process changes confirm, raising Web tool use by 8.0%.
University of Science and Technology of China, Qwen Business Unit of Alibaba · Oct 2026
- Big Techcs.CV
Edge vision-language models are as physically attackable as classic object detectors
Edge grounding VLMs are as vulnerable to physical adversarial patches as object detectors, and often fail by returning confident but wrong boxes, not omissions.
Saratoga High School, Google · Oct 2026
- Big Techcs.SE
Code understanding, not edit volume, limits coding agents
Code understanding is the bottleneck for coding agents: synthetic tasks show tools mask other LLM weaknesses, and understanding effort predicts errors better than edit count.
Microsoft Research, University of Maryland · Oct 2026
- Chinese Techcs.SE
Coding agents fully deliver enterprise PRD requirements only 14 percent of the time
Coding agents often implement individual pieces but rarely complete enterprise PRDs, with missing functionality and integration as the main failure modes.
Zhejiang University, Alibaba Taobao & Tmall Group · Oct 2026
- Chinese Techcs.CL
Adaptive training harness keeps rubric rewards useful as policies improve
HarnessRL co-evolves task sampling, rollout guidance, and rubric criteria during RL training, beating baselines by 2.8-3.1 points on open-ended tasks.
Alibaba Token Hub, Alibaba Group, Université de Montréal · Oct 2026
- Research Labcs.AI
Execution traces enable precise localized repair of agent memory architectures
Localized code repair using raw traces and memory state improves agent memory architectures more than coarse feedback.
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · Oct 2026
- Top Universitycs.LG
Internal representation shifts can expose multi-turn attacks as they emerge
Multi-turn attacks on LLM agents leave detectable signatures in internal representation transitions across context updates.
Singapore Management University, National University of Singapore · Oct 2026
- Chinese Techcs.SE
Diagnostic tools speed agentic root cause analysis, but accuracy varies
Higher-level diagnostic tools reduce localization accuracy but improve failure type identification and speed in agentic RCA.
Nankai University, Huawei Technologies Co., Ltd. · Oct 2026
- Big Techcs.LG
Persistent evidence helps long-horizon agents recover from earlier mistakes
Persistent belief updates across rollback plus an implicit world model improve failure recovery in long-horizon agents, and distillation cuts teacher cost.
University of Maryland, Google · Oct 2026
- Big Tech
Production-scale AutoResearch exposes five failure modes and a three-principle fix
AutoResearch at production scale reveals five failure modes fixed by a prevent-persist-redirect multi-agent framework.
Amazon · Sept 2026
- Independentcs.AI
Whole-system training improves general-purpose agents’ use of external tools
WEFT scales tool-use post-training for general-purpose agents by coupling whole-system interaction construction, execution-driven self-evolution, and stable post-training.
Sept 2026
- Chinese Techcs.AI
Contrasting multiple trajectories sharpens skill learning for LLMs
SkillCome improves skill evolution for language models by contrasting multiple successful and failed trajectories and using dual memory to derive reusable skills.
Fudan University, Ant Group · Sept 2026
- Chinese Techcs.AI
Causal error graphs improve diagnosis of failures in agent traces
Causal Error Graphs link execution events, diagnostic nodes, and failure outcomes, making agentic trace diagnosis explicitly causal rather than anomaly-driven.
Beijing Institute of Technology, Zhipu AI · Sept 2026
- Independentcs.AI
Polished evidence makes LLM agents act on unknowable questions
Authoritative-looking fabricated evidence makes LLM agents commit to unknowable questions; the action gate can be fine-tuned.
Aug 2026
- Research Labcs.SE
Cross-task failure diagnosis makes LLM agent harness training faster
Ecdysis aggregates failure evidence across task instances to train runtime harnesses for LLM agents, improving reasoning accuracy by 18.56%.
Chengdu Institute of Computer Applications, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Sept 2026
- Independentcs.AI
Synthetic rewards train agents to diagnose simulated advertising anomalies
Reinforcement learning with synthesized rewards from a simulator enables a small language model to outperform larger frontier models at diagnostic reasoning.
Independent Researchers · Sept 2026
- Top Universitycs.AI
Adaptive trace graphs improve failure attribution in multi-agent systems
Adaptive Influence Graphs improve failure attribution in multi-agent LLM systems through structured trace representation.
Tel Aviv University, AWS Agentic AI · Aug 2026
- Big Techcs.CL
Benchmark errors inflate and deflate AI policy-reasoning scores
Policy-based decision benchmarks have defects that misjudge rule-following responses, impacting scores and comparisons.
Amazon Web Services, Iowa State University · Nov 2026
- Top Universitycs.AI
LLM pipelines can lose 40.5 accuracy points between stages
Multi-stage LLM pipelines can lose up to 40.5 accuracy points when intermediate stages cannot access the original problem, with re-grounding recovering much of the loss.
Rutgers University, Nanjing Tech University · Sept 2026
- Top Universitycs.AI
Mobile GUI agents rarely transfer task success to new apps with the same goal
Mobile GUI agents do not generalize across apps for the same task; app-specific success does not transfer.
KAIST, CMU · Oct 2026
- Academiccs.CR
Security agents lose evidence support when available telemetry changes
When telemetry changes, LLM agents lose citation support for 35.5% of previously covered attack actions even though the actions remain recoverable.
Zhongguancun Laboratory · Oct 2026
- Top Universitycs.RO
Coding agents control simulated robots less reliably under realistic constraints
The strongest coding agent succeeds only 34% of the time on robot tasks under real-world constraints.
Northwestern University, Carnegie Mellon University · Nov 2026
- Academiccs.LG
AI teacher adapts its hints as a language model learns, boosting performance across four domains
AutoScaffold uses a frozen teacher LLM to automatically design adaptive training scaffolds that improve language model RL performance across domains.
University of Minnesota, Twin Cities, JPMorgan AI Research · Nov 2026
- Big Techcs.CL
Coding agents improve more when their model and harness co-evolve
Alternating self-proposed harness revisions with policy reinforcement learning improves coding-agent resolve rates beyond optimizing either component alone.
Microsoft Research, University of Virginia · Oct 2026
- Chinese Techcs.AI
Protocol risks in multi-agent systems depend on downstream agent decisions, not just local exploitability
Protocol risk in multi-agent systems is determined by downstream agent decisions, not just local exploitability.
Ant Group, Shanghai Jiao Tong University · Oct 2026
- Top Universitycs.CR
Independent logs make deployed agent records verifiable after failures
Agent harness records are not independently auditable; an external append-only witness with bidirectional reconciliation detects omissions and fabrications that self-authenticated logs miss.
Nanyang Technological University, National University of Singapore · Sept 2026
- Chinese Techcs.AI
LLM agents degrade reliably under controlled interaction perturbations
Perturbations to LLM agent inputs consistently degrade performance, and agents rarely detect corrupted information.
The Hong Kong University of Science and Technology (Guangzhou), Tencent Hunyuan · Oct 2026
- Academiccs.CR
Security skills guide coding agents to only 11% fully correct vulnerability detection
Current security skills rarely make coding agents fully identify Linux kernel vulnerabilities, while failure-driven skill refinement substantially improves detection on unseen cases.
York University · Oct 2026
- Top Universitycs.SE
Coding agents struggle to turn software proposals into working changes
Coding agents resolve at most 14% of long-horizon enhancement proposal tasks on large software systems, and incomplete code localization is a major bottleneck.
Fudan University, City University of Hong Kong · Sept 2026
- Academiccs.SE
Expert verification failures become reusable guidance for coding agents
Expert diagnosis of verification failures can be captured as reusable guidance for coding agents.
University of Illinois Urbana-Champaign · Oct 2026
- Independentcs.AI
AI judges reverse correct verdicts after a false note
A false authority cue overturns up to 40% of a grounded AI judge's correct verdicts.
Nov 2026
- Big Techcs.AI
Graded curricula extracted from an environment's latent rules substantially improve trained agents
Gradus composes guarded behavioral operators into calibrated curricula, lifting a 30B model from 59% to 89% on tau2-Bench and 47% to 83% on CAR-Bench.
AWS AI, University of Southern California · Oct 2026
- Chinese Techcs.AI
Localized Context Distillation resolves the learning dilemma in interaction experience
Localized Context Distillation improves LLM learning from interaction experience by having a teacher regenerate only critical spans of each student trajectory.
Tencent, The Chinese University of Hong Kong · Oct 2026
- Big Techcs.MA
Enterprise agent risks emerge when local decisions break shared constraints
EAIR provides the first framework for diagnosing and repairing interaction failures in enterprise multi-agent systems.
LMU Munich, Microsoft · Oct 2026
- Big Techcs.AI
Turning failed reasoning attempts into training signal improves GRPO on hard problems
Failed reasoning attempts can be turned into verified supervision for GRPO by having the policy write corrective notes and distilling only replays that beat the original group.
University of California, San Diego, Adobe Research · Oct 2026
- Top Universitycs.SE
Coding agents nearly finish repositories but still miss explicit requirements
The strongest coding agent solved only 10 of 11 repository construction tasks, with failures due to single specification omissions or low-frequency rules.
Gradient Data, McGill University · Oct 2026
- Big Techcs.AI
Verified self-improvement makes agent harness optimization more reliable
VERSE improves harness optimizers by letting them test edits and evolve their own verification tools.
MIT, Amazon · Oct 2026
- Top Universitycs.LGcode
Evidence-driven harness edits improve frozen GUI agents across benchmarks
GUI-HARVEST automatically improves GUI agent harnesses by diagnosing failures from repeated executions and generating reusable edits, boosting performance across models.
The Chinese University of Hong Kong, Shenzhen, Tianjin University · Oct 2026
- Big Techcs.AI
Training agents around decisive errors improves prevention and recovery
PivotOPD jointly trains agents to prevent and recover from pivotal mistakes, achieving top average performance across three benchmarks.
Princeton University, NVIDIA · Oct 2026
- Chinese Techcs.CL
Self-diagnosis pinpoints which agent decisions deserve reinforcement learning credit
In agentic RL, self-diagnosed errors turned into step-level credit anchored by terminal outcomes recover learning signal from same-outcome groups.
Alibaba Group, Kyoto University · Oct 2026
- Big Techcs.RO
Robots improve reusable manipulation skills through guided simulation practice
RPG improves robot task success from 28.6% to 95% through autonomous practice without updating model weights.
UC Berkeley, MIT · Oct 2026
- Chinese Techcs.RO
Robots recover more often when vision-language models monitor failures
Spotter improves robot task success by letting embodied policies act continuously while a parallel vision-language model detects, diagnoses, and repairs failures.
Griffith University, IIIS, Tsinghua University · Sept 2026
- Big Techcs.AI
Small surrogate models can flag errors in black-box LLM agents
A small surrogate model's log-probabilities audit black-box LLM agent actions better than the agent's own confidence or self-consistency.
Amazon · Oct 2026
- Big Techcs.SE
Coding agents' unnoticed mistakes make later requests fail more often
Coding agents left to run a chain of requests unsupervised finish about half of them, not because their code is poor but because they cannot spot their own mistakes.
Amazon · Nov 2026
- Research Labcs.CL
Individually approved agent skills can combine to break workflow policies
Individually safe agent skills can combine to cause policy violations, and adding a few benign skills nearly doubles the violation rate.
University of Science and Technology Beijing, ZODA · Oct 2026
- Top Universitycs.AI
Language models' own hallucinated text can reveal which memories to erase
Hallucination traces reveal overfitted memories, enabling targeted unlearning that balances forgetting efficacy and general utility without retraining.
Peking University, University of Technology Sydney · Nov 2026
- Chinese Techcs.AI
Coding agents struggle with infrastructure changes that existing tests miss
InfraDev shows top coding agents succeed on only 38.9% of realistic AI infrastructure tasks.
University of Edinburgh, Tencent Hunyuan · Oct 2026
- Top Universitycs.AI
Formal plan validation shows LLM agents rarely plan as claimed
LLM agents frequently violate action preconditions, rarely identify impossible goals, and gain little from elaborate prompting or agent architectures compared with changing the underlying model.
Barcelona Supercomputing Center, University of Amsterdam · Oct 2026
- Academiccs.LG
AI agents repeat targeted mistakes about half as often after retraining
MiRL improves terminal agents by training from recurring failure points and rewarding rollouts that both pass tests and avoid the targeted misbehavior.
University of North Carolina at Chapel Hill · Oct 2026
- Top Universitycs.AI
An agentic framework that diagnoses and verifiably repairs foundation model failures
EvalRx, an open-source agentic framework, statistically diagnoses and verifiably repairs foundation models, improving all 12 evaluated model-modality combinations by +1.4 to +10.9 percentage points.
The University of British Columbia, Vector Institute · Oct 2026
- Big Techcs.MA
Modeling multi-agent LLM systems as dynamic systems for root-cause analysis
AgentSoV models LLM multi-agent system execution as a stochastic dynamic system for failure attribution, influence analysis, and sensor placement.
University of Washington, Texas A&M University · Oct 2026
- Chinese Techcs.CL
Contrasting group executions pinpoints which agent skill caused success or failure
GRSE compares multiple executions of the same question to assign credit to the responsible agent and skill, outperforming prior methods.
Mayo Clinic, University of Manchester · Oct 2026
- Chinese Techcs.CL
Language models’ self-assessments can look accurate without predicting future performance
Language models can report supplied statistics exactly yet still fail to build trustworthy domain-level self-assessments from noisy user feedback.
The Chinese University of Hong Kong (Shenzhen), Shenzhen Research Institute of Big Data · Oct 2026
- Independentcs.CV
Leading AI models misread images in roughly 85% of failures
Most vision-language model failures come from misperception, not faulty reasoning, and accuracy alone hides this.
Oct 2026
- Chinese Techcs.SE
Coding agents struggle to extend features built on their own code
Coding agents struggle to maintain correct features across multiple stages, with whole-feature success at 28.5%.
Hunyuan Team, Tencent, The Chinese University of Hong Kong · Oct 2026
- Chinese Techcs.CV
Single-image 3D generation creates assets that move and deform realistically
PhysAg3D generates interactive 3D assets from a single image by coupling a 3D generator with a vision-language agent.
Peking University, Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) · Oct 2026
- Big Techcs.CV
Explicit goal grounding helps navigation agents stop at objects
Object-goal navigation policies fail mainly at goal grounding, not exploration; explicitly grounding the goal from scene labels substantially improves success.
Amazon FAR, New York University · Oct 2026
- Chinese Techcs.SE
Coding agents rarely fix failures within two rounds of corrections
FeatureLoop benchmarks coding agents on evolving requirements; even the best models need repeated user corrections and frequently leave requirements unfinished.
Hunyuan Team, Tencent, The Chinese University of Hong Kong · Oct 2026
- Top Universitycs.AI
Faster Jev decisions do not consistently improve agent task efficiency
Jev cuts decision cost and latency, but workflow overhead and failures erase gains, reducing mobile-agent success and sometimes raising cost per successful outcome.
Nanjing University, Technical University of Munich · Oct 2026
- Big Techcs.AIcode
AI agent teams coordinate worse than a single shared agent
Multi-user multi-agent teams deliver worse group outcomes than a single coordinator agent across four environments.
Independent Researcher, Stanford University · Oct 2026
- Top Universitycs.AIcode
Agent performance depends on matching models, harnesses and tasks
A model's performance and ranking depend on both the harness and the task.
Nanyang Technological University · Oct 2026
- Academiccs.LG
Spectral theory diagnoses two failure modes in RoPE long-context models
RoPE's high-frequency components cause a tradeoff between semantic stability and positional sensitivity that can be measured and mitigated by frequency rescaling without training.
Independent Researcher, University of Illinois at Urbana-Champaign · Oct 2026
- Top Universitycs.AIcode
More Harness Programs Do Not Necessarily Produce Useful Specialization
Repeatable score gains from different LLM harness programs are dominated by persistent weaknesses, not useful specialization.
The Chinese University of Hong Kong, New Laboratory of Pattern Recognition (NLPR) · Sept 2026
- Top Universitycs.SE
LLM agents deploy repairs better than systems built from scratch
FDE-Bench reveals that readiness checks are the primary failure point for LLM agents in deployment tasks.
University of California, Berkeley, Imperial College London · Sept 2026
- Big Techcs.AI
Adaptive training curricula produce stronger optimized agent harnesses
ActiveSaddler adaptively selects training scenarios for harness optimization, raising Pass@1 by 4.4 and 7.5 points on GAIA2 and Terminal-Bench 2.0.
POSTECH, KAIST · Oct 2026
- Research Labcs.AI
Agents Learn Reusable Control Code Instead of Rebuilding Context
Growing Harness learns reusable agent control code from task failures, reducing LLM calls by up to 91.8% while maintaining or improving task success.
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Sept 2026
- Chinese Techcs.LG
Refining rejected candidates makes AI system optimization more efficient
Mara Chain retains and refines rejected AI system candidates instead of discarding them, improving performance with fewer evaluations.
Ant Group, Beijing Intelligent Game and Decision Lab · Sept 2026
- Academiccs.AI
Framework improves AI agents by patching common failure patterns
Vestrum turns execution trace failures into scoped harness changes that improve performance on held-out tasks without retraining the model.
University of Georgia · Oct 2026
- Top Universitycs.AI
Agents improve long-horizon performance by evolving how they manage context
Evolving context management policies from long-horizon trajectories improves long-horizon agent performance more effectively than evolving skills or using fixed context strategies.
Fudan University, Shanghai Key Laboratory of Data Science · Sept 2026
- Big Techcs.AI
Agentic data synthesis improves by evolving the generation space based on learner feedback
EvoSyn evolves a revisable synthesis space using learner feedback, reallocating and expanding generation controls, and improves agent task success by 13.3 points.
University of Southern California, AWS AI · Oct 2026
- Academiccs.AI
AI agents reproduce only 41% of machine learning papers with full code and weights provided
AI agents reproduce ML paper results at 15-41% rates depending on code availability; most failures lack verification against the paper's numbers.
University of Illinois Urbana-Champaign, National Center for Supercomputing Applications · Oct 2026
- Big Techcs.AI
Terminal agents struggle when familiar environmental assumptions stop holding
LLM agents fail to adapt when environmental assumptions change, but training on such novelties improves both adaptation and base task performance.
University of Illinois Urbana-Champaign, Microsoft Research AI Frontiers · Sept 2026
- Research Labcs.AI
Diagnostic feedback drives iterative benchmark evolution revealing hidden LLM agent weaknesses
EvoProbe uses fine-grained diagnostics to iteratively generate harder benchmark tasks that probe LLM agent capabilities.
National University of Defense Technology, Institute of Information Engineering, Chinese Academy of Sciences · Oct 2026
- Top Universitycs.AI
Reusable harness components let agents adapt workflows to each task
Selecting and composing reusable harness mechanisms at test time improves agent task success rates by up to 12 points.
University of Illinois Urbana-Champaign, University of Michigan, Ann Arbor · Oct 2026
- Big Techcs.AI
Long context windows do not ensure reliable sustained execution
Long-Transduction reveals that models suffer 62.8% accuracy loss when context length scales to 128K tokens in long-horizon agentic workflows.
NVIDIA · Oct 2026
- Big Tech
Formal audit detects omitted source relationships in generative search
A four-predicate audit can determine whether a generative search answer omits a source relationship that changes its interpretation.
Google LLC, Sony Corporate of America · Sept 2026
- Big Tech
Evolved Agent Harnesses Improve Coverage but Fail to Consolidate Gains
Automatically evolved LLM agent harnesses improve completion and task coverage in hardware verification, but complementary gains do not reliably consolidate into one consistently superior harness.
NVIDIA · Sept 2026
- Academiccs.CRcode
Cyber agents hide critical weaknesses behind successful attack workflows
Multi-stage LLM cyber agents face persistent bottlenecks in credential and lateral-movement tasks that success rates alone do not reveal.
National Research Council, University of Windsor · Sept 2026
- Top Universitycs.AI
Agent safety monitors struggle to intervene before multi-step risks escalate
Proactive intervention in multi-step LLM agent trajectories remains largely unsolved; the best evaluated model intervenes at the optimal time only 40.74% of the time.
The Hong Kong University of Science and Technology, Peking University · Sept 2026
- Big Techcs.MA
Counterfactual world models improve fully decentralized multi-agent learning
Agents learn more effectively by prospectively comparing consequences of candidate actions using counterfactual world models.
Fordham University, City University of Hong Kong · Oct 2026
- Big Techcs.AI
Retrieving existing agent skills improves optimization across different execution harnesses
RASO leverages an external skill corpus through initialization and update, adapting retrieved skills across harnesses, and outperforms retrieval-free baselines on four benchmarks.
Korea University, KAIST · Sept 2026
- Industrycs.CL
Tool-call traces expose extraction failures that source fidelity misses
Agentic extraction requires dispatch-level instrumentation to detect silent failures that fidelity checks miss.
Infineon Technologies AG · Sept 2026
- Big Techcs.AI
Memory guidelines help language-model agents succeed consistently across repeated runs
LLM agents averaging 77% per-run success on AppWorld pass all five runs on only 53% of tasks; self-evolving consistency guidelines raise this by 16 points same-task and 13 similar-task.
IBM Software Innovation Lab, IBM Research · Sept 2026
- Academiccs.AI
Joint training helps small language models create and use tools
SMITH jointly optimizes tool creation and use via RL, achieving 79.8% accuracy on held-out reasoning tasks.
Appier AI Research, National Taiwan University · Aug 2026
- Independentcs.AIcode
Good tool plans can still fail under resource constraints
PeakBench benchmarks LLM agents on resource-aware tool scheduling, showing logical planning doesn't guarantee safe or efficient parallel execution.
Aug 2026
- Big Techcs.CL
Structural analysis filters noisy agent traces to pinpoint root causes
STRACE filters redundant trajectories and extracts causally relevant steps from execution traces, improving agent optimization with a 1.4x success-rate gain on formal verification.
University of Chinese Academy of Sciences, Microsoft Research · July 2026
- Big Techcs.AI
Self-supervised method improves agent harnesses using past trajectories
Retrospective Harness Optimization improves AI agent harness using past trajectories and self-preference, boosting SWE-Bench Pro pass rate from 59% to 78% without external labels.
City University of Hong Kong, Microsoft Research Asia · June 2026
- Academiccs.CL
Serving stacks can distort local language-model tool-use evaluations
Local serving stacks can confound coding agent tool-use evaluation, making measurements reflect the stack rather than the model.
Northeastern University, Seattle · Sept 2026
- Big Techcs.CL
Process-based evaluation reveals where computer-use agents go wrong
OSWorld-Pro evaluates computer-use agents on intermediate subgoals to reveal specific failure modes.
NVIDIA · Sept 2026
- Big Techcs.CL
Extra attempts help small language models solve terminal tasks, but not verify solutions
Small language models fall behind frontier models on terminal tasks not only in solving them but also in verifying solutions and anticipating their own success.
Microsoft Research, University of California, Los Angeles · Oct 2026
- Big Techcs.SE
Governing agents' evidence with deterministic harness rules beats free-form trajectories
Agentic Interpretation uses harness-governed evidence construction to improve agent patch acceptance and fault localization.
University of California, Davis, AWS AI Labs · Oct 2026
- Top Universitycs.RO
Agent-side memory steers stateless robots through long manipulation tasks
Task memory for long-horizon manipulation can live outside the action policy; an agent-steered, stateless RGB action model reaches 76.3% on LIBERO-Mem.
KU Leuven, Meituan Inc. · Sept 2026
- Independentcs.AI
Game-theoretic filtering helps agents suppress harmful memories during long tasks
Long-horizon LLM agents can selectively suppress harmful retrieved memories via cooperative Shapley attribution, achieving 85.9% target recall and 82.3% task recovery.
Sept 2026
- Big Techcs.AI
Skill evolution improves agent performance in image generation workflows
Self-evolving skill libraries improve agent reliability in recurring image generation workflow construction.
University of Pennsylvania, Nvidia · July 2026
- Chinese Techcs.AIcode
Evolutionary training harness co-evolves with LLM policies for RL
Co-evolving LLM policies with training-side diagnostic harnesses matches or exceeds human-engineered RL references across math, coding, and long-horizon software engineering.
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Tongyi Lab , Alibaba Group · June 2026
- Big Techcs.AI
Memory as state management instead of semantic retrieval improves long-horizon agents
MAGE improves task success rate by 7.8–20.4 percentage points and reduces token consumption by 55.1% on MemoryArena.
University of Science and Technology of China, Microsoft · June 2026
- Academiccs.SE
Tracing and fault injection make multi-agent software workflows easier to inspect
llmmas-otel combines OpenTelemetry tracing with fault injection to inspect and test LLM-based multi-agent systems for software engineering.
Delft University of Technology, JetBrains Research · Aug 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.