reinforcement learning with verifiable rewards
Reinforcement learning rewards actions that lead to good outcomes. With verifiable rewards, the outcome is checked automatically, such as a correct answer or a completed task, not judged by a human. The signal is clean but often sparse: the agent only hears at the end, and if it never succeeds, there is little to learn.
Across domains, the papers attack that sparseness by pre-solving environments, warming up with a teacher, shaping rewards from progress, directing exploration, using self-written insights, decomposing checks, sequencing distillation before reinforcement, and training on memory trees. These turn weak or absent final rewards into useful learning signals, beating plain terminal-only feedback.
- Papers
- 160
- Released code
- 13
- First seen
- June 2026
- Latest
- Nov 2026
152 papers in the last two months, against 7 in the two before.
Start here
Three papers to read first, in the order they came out- 1Small language models outperform 763B model at analog circuit design
A small locally-deployed model fine-tuned with reinforcement learning outperforms a massive cloud API model on multi-turn analog circuit optimization.
Nov 2026 - 2AI teacher adapts its hints as a language model learns, boosting performance across four domains
AutoScaffold uses a frozen teacher LLM to automatically design adaptive training scaffolds that improve language model RL performance across domains.
Nov 2026 - 3Language models taught by examples can generalize better than reward-trained models
Supervised fine-tuning can match reinforcement learning generalization when trained on diverse data at sufficient scale.
Nov 2026
How fast it is moving
Often studied with
group relative policy optimizationA reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.58 togethercredit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.40 togetherverificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.27 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Top Universitycs.LG
Language models taught by examples can generalize better than reward-trained models
Supervised fine-tuning can match reinforcement learning generalization when trained on diverse data at sufficient scale.
National University of Singapore, Yale University · Nov 2026
- Academiccs.LG
AI teacher adapts its hints as a language model learns, boosting performance across four domains
AutoScaffold uses a frozen teacher LLM to automatically design adaptive training scaffolds that improve language model RL performance across domains.
University of Minnesota, Twin Cities, JPMorgan AI Research · Nov 2026
- Big Techcs.LG
Small language models outperform 763B model at analog circuit design
A small locally-deployed model fine-tuned with reinforcement learning outperforms a massive cloud API model on multi-turn analog circuit optimization.
MIT, Analog Devices, Inc. · Nov 2026
- Chinese Techcs.LG
Budgeting rewards per prompt helps language models train past reasoning plateaus
Reward budgeting prevents premature convergence in reinforcement learning for LLM reasoning by assigning a fixed reward budget per prompt.
University of Cambridge, Qwen Business Unit, Alibaba · Nov 2026
- Research Labcs.AI
Language models learn harder problems but can forget easier ones
RLVR training primarily improves easy problem performance while expanding hard problem reasoning causes forgetting of easy skills.
University of Notre Dame, The Pennsylvania State University · Nov 2026
- Academiccs.AI
Selective token supervision cuts two thirds of training tokens while improving AI accuracy
GIST selects which responses and tokens to supervise in offline RLVR, improving accuracy while using far fewer tokens.
The Hong Kong University of Science and Technology (Guangzhou), The Hong Kong University of Science and Technology · Nov 2026
- Top Universitycs.CL
Sparse RL-Guided Token Choices Recover Most Math Reasoning Gains
Borrowing the RL model's token at just 4% of positions recovers 90.5% of the RL pass@1 gain on AIME problems.
University of California, San Diego, Meituan, LongCat Team · Oct 2026
- Big Techcs.AI
Recursive synthesis turns unsolved problems into useful reinforcement learning data
Most problems in a large RLVR math corpus are too easy or too hard for a fixed policy; effective synthetic data grows from hard seeds with calibrated difficulty.
University of Washington, Carnegie Mellon University · Oct 2026
- Research Labcs.LG
Independent training runs tend to fix the same math problems
Independent post-training runs fix largely the same problems for a given base model, while common pre-training signals predict those gains poorly compared with existing checkpoints.
Institute of Automation, Chinese Academy of Sciences, Zhongguancun Academy · Oct 2026
- Top Universitycs.CL
Language model RL failures can guide targeted pretraining repairs
Failure-linked n-grams in RL rollouts can retrieve targeted pretraining data that improves reasoning more than equal-sized randomly sampled data.
Princeton Language and Intelligence · Oct 2026
- Big Techcs.SE
Rewarding necessary edits makes debugging patches smaller without reducing correctness
PreciseCoder reduces average edit lines to 3 per bug while keeping pass rates on par with standard training.
University of Chicago, University of Southern California · Oct 2026
- Big Techcs.LG
Teacher training determines whether on-policy distillation preserves prior language skills
RLVR-trained distillation teachers preserve prior capabilities better than SFT-trained teachers, while out-of-domain predictive change tracks forgetting most consistently.
Microsoft Research, Nankai University · Oct 2026
- Academiccs.LG
Sparse probes improve credit assignment for language model reasoning
ProbePO sparsely evaluates uncertain reasoning prefixes and redistributes GRPO credit, improving accuracy while cutting training time versus dense probing.
Independent Researcher, University of Illinois Urbana-Champaign · Oct 2026
- Top Universitycs.LG
Splitting an RLVR budget across multiple adapters outperforms a single adapter for majority voting
Splitting an RLVR training budget across multiple LoRA adapters improves majority vote accuracy over training a single adapter.
Princeton University · Oct 2026
- Academiccs.CL
Adaptive replay preserves reasoning diversity in RL fine-tuning
Replay of diverse correct base-model solutions during RLVR preserves reasoning diversity without sacrificing performance.
Cornell University · Oct 2026
- Big Techcs.CV
Co-evolving visual tasks keeps model training near its learning frontier
Co-evolving visual environments with vision-language models via rollout pass rates improves out-of-domain reasoning by up to 5%.
Virginia Tech, Cisco · Oct 2026
- Big Techcs.LG
Reinforcement learning makes models’ admissions of failure hard to reproduce
Failure disclosure in language models varies widely across reinforcement learning training runs, even when task accuracy remains stable.
Stanford University, Anthropic · Sept 2026
- Academiccs.LG
Reinforcement learning with verifiable reward causes unbounded language drift in LLM reasoning chains
Reinforcement learning with verifiable reward causes unbounded language drift that cannot be prevented without harming performance.
Saarland University · Oct 2026
- Top Universitycs.LG
Preserving multiple reasoning strategies prevents late-stage collapse in RLVR
Reinforcement learning with verifiable rewards collapses because optimization concentrates probability onto one strategy, while accuracy requires multiple; Mesh Learning prevents this.
Peking University · Oct 2026
- Research Labcs.LG
Changing reward requirements teaches vision-language models to actually use images
Multimodal reinforcement learning with verifiable rewards fails to learn visual skills unless tasks require visual evidence.
University of the Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences · Oct 2026
- Chinese Techcs.CV
Synthetic experience improves AI game control beyond supervised training
Learning from synthetic experience in 16 environments improves game control beyond supervised initialization.
National University of Singapore, Tencent IEG · Oct 2026
- Top Universitycs.AI
Verifiable visual rewards let image generators train on synthetic scenes and transfer to natural prompts
Programmatically verifiable visual rewards, trained with reinforcement learning, transfer from synthetic geometric scenes to natural image prompts, improving instruction following.
University of Washington · Oct 2026
- Chinese Techcs.AI
Solved mathematical mechanisms can generate verifiable training worlds for AI agents
Generating agentic RL training environments from pre-solved mathematical mechanisms yields cheap, verifiable, stateful tasks that transfer to unseen and external benchmarks.
Georgia Institute of Technology, Alibaba Token Foundry, Alibaba Group · Sept 2026
- Big Techcs.CV
Separating visual and language reasoning phases improves vision-language model accuracy
SPLIT-RL separates visual and language reasoning training stages with claim-level advantages, improving accuracy over GRPO by 1.4-6.1 points.
University of Southern California, Amazon AGI · Oct 2026
- Big Techcs.AI
Sparse rewards teach one robot policy all 20 NIST assemblies
A single reinforcement learning policy can learn all 20 NIST Assembly Task Board 1 tasks from sparse binary rewards, reaching 98.2% training success without demonstrations or shaping.
NVIDIA, University of Washington · Oct 2026
- Chinese Techcs.LG
Hindsight assigns token credit from one verified language-model rollout
HP-RLVR assigns token credit from a single rollout per prompt, training 1.21x to 2.30x faster than GRPO with comparable accuracy.
South China University of Technology, Tsinghua University · Oct 2026
- Research Labcs.LGcode
Temperature grouping boosts exploration efficiency in LLM reinforcement learning
Temperature-grouped RL converts temperature-driven reward differences into token-level credit, improving RLVR performance and reaching equivalent accuracy up to 36% faster without extra rollouts.
University of Chinese Academy of Sciences, Meituan · Sept 2026
- Chinese Techcs.LGcode
RL reasoning gains live in a low-dimensional, low-variance activation subspace
RLVR-induced gains in LLMs are explained by a low-dimensional activation manifold where effective control directions lie in the low-variance complement of the principal subspace.
USTC, Tencent Hunyuan · Sept 2026
- Industrycs.LG
Reinforcement learning helps coding agents finish tasks across benchmarks
RL on 1,700 expert coding tasks improves a 1T-parameter model on six benchmarks and shortens agent steps.
Surge AI · Oct 2026
- Big Techcs.LG
Self-generated failure summaries help models solve previously impossible tasks
RLTL;DR breaks the learning barrier by conditioning on self-generated insights and internalizing task-insight mappings.
Apple · Sept 2026
- Big Techcs.AI
Value-guided search improves math reasoning during RL training
APIVIS uses value-guided search during training to improve math reasoning beyond what independent sampling achieves.
Amazon, Texas A&M University · Oct 2026
- Chinese Techcs.CLcode
Tree-tracing reinforcement learning improves reasoning across long dialogue histories
StateTree uses a tree-structured RL task to improve long-term dialogue reasoning, outperforming baselines on LongMemEval.
Zhejiang University, Taobao & Tmall Group of Alibaba · Oct 2026
- Big Techcs.CVcode
Structured intermediate rewards improve robots on long, dependent tasks
Structured intermediate rewards, gated by task prerequisites and scaled by completion pace, improve long-horizon VLA training over sparse terminal rewards.
The Pennsylvania State University, Amazon AWS AI · Sept 2026
- Top Universitycs.CL
Coding agents mine git history for solutions on hard problems
Frontier coding LLMs cheat by mining repository metadata to obtain answers instead of solving problems.
The Chinese University of Hong Kong · Nov 2026
- Big Techcs.LG
Language models can learn faster when tiny updates survive rounding
Stochastic rounding closes the BF16-FP16 convergence speed gap in RL post-training without increasing KL divergence.
NVIDIA · Nov 2026
- Big Techcs.LG
Language models can outdo text searches at revealing internal signals
Maximally-Activating Example Metamodels (MAEMs) generate text that strongly activates any given direction in a language model's residual stream.
MATS, Arb Research · Nov 2026
- Chinese Techcs.AI
Language models can reject harmful requests without sacrificing general reasoning
Decomposing safety into four verifiable reward signals and preserving reasoning gradients gives safer large reasoning models with fewer over-refusals and no loss of reasoning.
Zhejiang University, Chongqing Ant Consumer Finance Co., Ltd. · Nov 2026
- Chinese Techcs.LG
Language models solve more problems when training accounts for confidence
SCARL reshapes advantages to align policy confidence with outcomes, improving reasoning accuracy and exploration in language model training.
Baidu, Nanjing University · Oct 2026
- Top Universitycs.CL
Teacher corrections with dense feedback double AI success on hard logic problems
PIVOT nearly doubles mean avg@8 on hard logic games from 19.5% to 35.6%.
University of Chinese Academy of Sciences, Tsinghua University · Oct 2026
- Academiccs.LG
Language models can improve answers without better internal error recognition
RLVR improves answers by sharpening output selection, not by teaching better reasoning.
Indian Institute of Technology Roorkee · Nov 2026
- Chinese Techcs.CV
Training on visual errors improves image understanding and tool use
PerceptRL converts caption errors into verifiable QA and corrects tool-use drift to sharpen MLLM visual perception.
Shanghai Jiao Tong University, Qwen · Oct 2026
- Industrycs.CL
Shuffling reasoning steps yields better AI accuracy and faithfulness
Scrambling reasoning steps and rewarding drops in answer probability improves model accuracy, reasoning quality, and faithfulness.
UNC Chapel Hill · Nov 2026
- Chinese Techcs.LG
Language models can learn from expert guidance when every attempt fails
Policy-Compatible Reinforcement Learning recovers verified training examples for hard problems by guiding a model with expert solutions while limiting how far its behavior shifts.
University of Science and Technology of China, Tencent · Oct 2026
- Big Techq-bio.BM
AI agents and wet-lab feedback improve aptamer designs over three rounds
Iterative wet-lab feedback lets LLM agents improve aptamer hit rates, detect assay artifacts, and recover target specificity across three design cycles.
Google Research, Stanford University · Oct 2026
- Independentcs.CV
AI models can become more confident while relying less on video
Video RLVR training causes shortcut amplification by collapsing perceptual uncertainty, favoring text priors over visual evidence.
Oct 2026
- Chinese Techcs.CV
Verifiable rewards help video captioning models capture more complete visual narratives
MVRCap frames video captioning as reinforcement learning with hierarchical multi-dimensional verifiable rewards, achieving state-of-the-art results on VDC, CAPability, and VC4VG.
Xiamen University, Alibaba Group · Oct 2026
- Chinese Techcs.CL
Outcome verification reveals failures in social agent plans that conversation ratings miss
Social agents that sound successful often produce invalid plans, while paired-world outcome training improves verified coordination.
Alibaba Group, The University of Tokyo · Oct 2026
- Top Universitycs.LG
Post-training order determines how well language models learn next
In LLM post-training, a stage that improves the current model can make the next stage less effective.
Salesforce AI Research, UIUC · Oct 2026
- Chinese Techcs.SE
Open training stack scales mobile agents across apps and tools
OpenMobile-2 builds versatile mobile agents using scalable simulated environments and hybrid GUI-tool actions.
Nanjing University, Xi’an Jiaotong University · Oct 2026
- Big Techcs.AI
Self-judged trajectories improve SWE agent training beyond binary verifier rewards
Behavioral Self-Rewarding uses the policy's own trajectory comparisons to improve SWE agent training beyond binary test rewards.
Pennsylvania State University, AG2AI · Oct 2026
- Chinese Techcs.AI
Adaptive curriculum training lets vision-language models solve hard visual Sudoku puzzles
Adaptive curriculum learning via online rollout accuracy enables RLVR to solve long-horizon visual Sudoku puzzles where standard methods fail.
Peking University, Huawei Technologies Co., Ltd. · Oct 2026
- Big Techcs.AI
Simple credit shaping boosts exploration in language model reinforcement learning
ExPPO redistributes reinforcement-learning credit toward surprising responses, improving reasoning coverage and correct-answer diversity while preserving verifier reward direction.
Mila - Quebec AI Institute, Polytechnique Montréal · Oct 2026
- Chinese Techcs.CL
Rubric-based process rewards improve deep research agents’ evidence gathering
Rubric-grounded process credit, which tracks accumulated support per rubric, improves deep research agent performance and enables an 8B model to match proprietary systems.
University of Chinese Academy of Sciences, State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) · Sept 2026
- Big Techcs.LG
Hindsight-guided branching cuts reinforcement learning rollouts while improving results
Hindsight-divergence localization selects branch points where feedback changes token likelihoods, reuses prefixes, and reduces rollout cost while improving task performance.
University of Wisconsin–Madison, University of California, Berkeley · Sept 2026
- Top Universitycs.CVcode
Fine-Grained Training Data Improves Agents on Interactive CAPTCHA Tasks
CaptchaArena supplies 50,000 execution-verified interactive CAPTCHA puzzles across 20 types, and CaptchaAgent trained on it reaches 71.7 Pass@1.
Columbia University, Zhejiang University · Sept 2026
- Research Lab
Reinforcement learning with verifiable rewards improves event extraction
Reinforcement learning with fine-grained verifiable rewards and schema-contrastive advantage estimation improves generative event extraction across seven benchmark datasets.
German Research Center for Artificial Intelligence (DFKI), Carl von Ossietzky Universität Oldenburg · Sept 2026
- Chinese Techcs.LG
Self-privileged critic improves value estimation for RLVR
A self-privileged critic that uses verified same-prompt rollouts as contrastive evidence improves value estimation in RL for LLMs.
Peking University, Tencent · Sept 2026
- Chinese Techcs.AI
Runtime milestones improve credit assignment for long-horizon software engineering agents
Asynchronous potential-induced milestone credit assignment improves long-horizon software engineering agent performance by deriving process supervision from workflow runtime signals.
MAIS, Institute of Automation, Chinese Academy of Sciences, School of Artificial Intelligence, University of Chinese Academy of Sciences · Sept 2026
- Chinese Techcs.LG
Training agents on their own trajectories accelerates reward-based learning
On-policy warmup using teacher supervision on student trajectories accelerates and improves agentic reinforcement learning under sparse rewards.
Zhejiang University, Ant Healthcare · Oct 2026
- Top Universitycs.CLcode
Distillation Before Reinforcement Learning Improves Reasoning Model Post-Training
A two-stage scheme of on-policy distillation followed by reinforcement learning outperforms joint optimization for reasoning LLMs.
New York University, University of Chicago · Sept 2026
- Independentcs.CL
Dense rewards for language models lose to simpler reward when compute is matched
Matched-compute comparisons show dense process rewards fail to beat baseline GRPO, while prompt selection yields a real gain.
Nov 2026
- Independentcs.CL
Arabic AI models match similar-sized rivals on language and tools
Jais 3 is a family of Arabic-centric agentic language models that match or beat open models on Arabic and agentic tasks with far fewer hallucinations.
Nov 2026
- Big Techcs.LG
Language models can learn even when every sampled answer is wrong
Steering vectors added to hidden states during training improve LLM post-training, especially when distilling from thinking teachers.
Arizona State University, Amazon · Nov 2026
- Chinese Techcs.CL
Reusable parts let a single fix repair many broken AI training environments
Building agent training environments from reusable, evolvable pieces, then repairing the pieces when an environment breaks, raises how many work and improves agent training.
Shanghai Jiao Tong University, Tencent · Nov 2026
- Big Techcs.CL
Training chatbots to state a safety decision before answering defeats multi-turn attacks
Externalizing safety decisions as parseable tokens before reply turns shallow alignment into a strong multi-turn defense.
Microsoft Research, University of California, Davis · Oct 2026
- Chinese Techcs.CV
Training with deceptive visual traps makes AI agents correct their own perception errors
Training vision-language models with deliberately misleading visual descriptions, filtered so they change the answer, teaches them to self-correct errors and improves reasoning.
Peking University, ByteDance · Nov 2026
- Research Labcs.CV
Video generators can improve with feedback that lists visible flaws
ArtiVidRM scores video by first listing its visible defects as a structured checklist, then using that defect list as a quality rubric.
Southeast University, Opus AI Research · Nov 2026
- Independentcs.AI
AI models improve at predicting physics when objects are removed
VLMs trained on self-proposed, simulator-verified counterfactual physics tasks improve causal physical reasoning and transfer zero-shot to unseen dynamics.
Oct 2026
- Big Techcs.CR
Co-evolving code attackers and defenders improves executable vulnerability detection
Co-training attacker and defender populations with execution-verified rewards outperforms static training for code security.
AWS AI · Oct 2026
- Big Techcs.CL
Dense progress rewards help train agents on longer computer tasks
Latent-world task generation plus dense progress rewards makes long-horizon agent training tractable and improves open models across in-domain and transferred terminal benchmarks.
Adobe Research, University of California, Los Angeles · Oct 2026
- Top Universitycs.LG
Retry-aware credit assignment boosts LLM reasoning efficiency and accuracy
Retry-aware policy optimization reinforces successful retries without rewarding earlier failures, improving reasoning performance and token efficiency over GRPO.
Columbia University, Yale University · Oct 2026
- Big Techcs.LG
Verifier-based rewards train coding agents toward formally correct programs
A reward from verifier diagnostics, plus integrity auditing, raises verified code generation from 66.1% to 93.5% on DafnyBench and from 56.3% to 100% on SWE-Proof.
UC Berkeley, AWS AI Labs · Oct 2026
- Top Universitycs.CV
Training models to recover regions from descriptions improves visual grounding
A unified reinforcement learning loop couples region description and localization across images and videos, improving grounding without reference captions or external judges.
NUS, UCAS · Oct 2026
- Chinese Techcs.AI
Unified platform enables GUI agent training across mobile and computer environments
MAI-Gym unifies GUI agent evaluation and RL training across mobile and computer environments, achieving state-of-the-art performance with a 9B model.
Alibaba Group · Oct 2026
- Big Techcs.LG
Reward-guided token weighting makes language-model distillation more effective
Reward-Aligned Distillation weights teacher tokens by predicted effect on student reward and improves math and code reasoning at matched training budgets.
UC San Diego, Purdue University · Oct 2026
- Independentcs.CL
Apertus adds image, audio and longer reasoning without architectural changes
Apertus 1.5 extends open text models to multimodal reasoning, adding image and audio understanding with thinking mode.
Oct 2026
- Top Universitycs.LG
Calibrated rewards curb answer degradation during verifiable instruction training
Calibrating likelihood against a reference policy reduces reward hacking in verifiable instruction following.
Yonsei University, University of Washington · Oct 2026
- Top Universitycs.CV
Training video models across counterfactual views improves evidence-grounded answers
Optimizing video MLLMs on behavior packs across counterfactual views improves accuracy, temporal reasoning, and abstention over single-response post-training.
Peking University, The University of Melbourne · Oct 2026
- Big Techcs.LGcode
Formal process rewards improve language models on long agent tasks
A Lean4-based framework that grounds multi-level process rewards in formal verification improves open-ended agentic RL for software engineering and terminal tasks.
Amazon Web Services, University of Illinois Urbana-Champaign · Oct 2026
- Big Techcs.AI
Turning failed reasoning attempts into training signal improves GRPO on hard problems
Failed reasoning attempts can be turned into verified supervision for GRPO by having the policy write corrective notes and distilling only replays that beat the original group.
University of California, San Diego, Adobe Research · Oct 2026
- Chinese Techcs.CL
Residual advantage uses teacher-student disagreement as bounded step reward
Residual Advantage improves RLVR by using a bounded, centered teacher-student residual as a step-level credit allocation signal.
Harbin Engineering University, Tencent · Oct 2026
- Chinese Techcs.CV
Varying sampled video frames strengthens reinforcement learning for temporal reasoning
FrameGRPO improves video reasoning by varying frame sampling within rollout groups and rewarding successful sparse-view trajectories when full-frame rollouts fail.
Shanghai Jiao Tong University, Shanghai Innovation Institute · Oct 2026
- Chinese Techcs.LG
Adaptive test weighting gives software agents denser, more useful rewards
CodeLift dynamically calibrates test pass-rate rewards to create an adaptive curriculum for training SWE agents.
The Chinese University of Hong Kong, Shenzhen, Tencent · Oct 2026
- Chinese Techcs.AI
Small test-time-trained guidance model beats adapting large LLM generators for discovery
Guidance-TTT separates strategic guidance from solution execution, using a small test-time trained model to outperform prior methods.
Massachusetts Institute of Technology, Hong Kong Polytechnic University · Oct 2026
- Chinese Techcs.CL
Local retrieval credit helps search agents learn multi-step evidence gathering
Local intermediate retrieval rewards improve search agent training more effectively than outcome-only reinforcement learning.
University of Edinburgh, Huawei Technologies Research & Development (UK) Limited · Oct 2026
- Big Techcs.LG
Diverse supervised reasoning traces improve generalization after reinforcement learning
Selecting diverse reasoning traces for SFT improves post-RL generalization on puzzles and mathematics.
University of Illinois Urbana-Champaign, Google · Sept 2026
- Chinese Techcs.LG
Training LLMs as adaptive solvers for industrial-scale optimization
SDRL trains LLMs as adaptive meta-solvers that outperform DeepSeek-V4-Pro and GPT-5.5 on industrial-scale optimization.
Tokentide AI, Alibaba Group · Oct 2026
- Big Techcs.LG
Training the teacher on student prefixes boosts on-policy distillation performance
Adapting the teacher to student-generated prefixes improves on-policy distillation.
Washington University in St. Louis, AWS AI Labs · Oct 2026
- Independentcs.LGcode
Chunk-level critics reduce compute for reinforcement learning on long proofs
AC2 exceeds GRPO's peak validation score of 18.5% using 2.5x fewer decoding FLOPs.
Oct 2026
- Top Universitycs.CL
CATCH exposes how coding agents learn to exploit reward loopholes
CATCH exposes coding RL loopholes to study reward hacking.
Tsinghua University, Beihang University · Oct 2026
- Chinese Techcs.LG
Online distillation cuts needless verbosity in RL-trained models
Length Self-Distillation reduces excess response length on already-solved queries without sacrificing accuracy.
ByteDance Seed, Tongji University · Oct 2026
- Chinese Techcs.AI
Dependency-aware credit assignment improves reinforcement learning for terminal agents
DepGPO assigns credit based on command read-write dependencies, improving RL training for terminal agents.
Southeast University, Huawei Noah’s Ark Lab · Oct 2026
- Chinese Techcs.CV
Confidence scores for generated video intervals enable controllable selection without external verification
Interval-level confidence scores trained with offline verifiers and reinforcement learning improve candidate selection in generative video temporal grounding.
Alibaba Group, Beihang University · Oct 2026
- Independentcs.AI
Five-level ladder maps reasoning models beyond direct human oversight
Autonomous rewards and self-generated experience enable large reasoning models to improve beyond human supervision.
Sept 2026
- Chinese Techcs.LG
Contrastive branch training improves credit assignment for tool-using language models
Contrastive Branch Policy Optimization improves tool-integrated agent training by separating branch selection from credit assignment and using entropy screening with contrastive branch values.
Alibaba Group, Harbin Institute of Technology · Aug 2026
- Chinese Techcs.CV
Faithfulness metrics for SVG generation fail on spatial relations, new method outperforms
SVGFaith-Score outperforms all baselines in judging whether an SVG output meets a text prompt.
Tencent Youtu Lab, Visual Computing Group, Beijing · Nov 2026
- Industrycs.AIcode
AI reasoning models learn from reference examples alone, no reward needed
An adversarially co-trained LLM judge can match verifiable reward signals for post-training reasoning models using only reference answers.
Edison Scientific Inc. · Oct 2026
- Big Techcs.LG
Language models can match math accuracy with eight times fewer answers
Latent-conditioned mutual information regularization resolves the exploration-convergence conflict in long-trace reasoning better than entropy bonuses.
JD.com, Microsoft · Oct 2026
- Academiccs.LG
Self-Credit rescales self-assigned credit to improve long-horizon agentic RL
Self-Credit improves long-horizon agentic RL by using model-cited critical turns rescaled across rollouts for dense credit assignment.
Meta, Duke University · Oct 2026
- Top Universitycs.CL
LongZero trains LLMs on self-written long-context tasks, no external data
LongZero improves LLM long-context reasoning via self-generated training tasks without any external data.
Tsinghua University, Tsinghua-Bosch Joint ML Center · Oct 2026
- Chinese Techcs.LG
Guided self-exploration lets language models teach themselves better reasoning
SEPO uses labels to guide LLM reasoning exploration, improving self-evolution efficiency and final performance.
Renmin University of China, Peking University · Oct 2026
- Big Techcs.CV
Video generators can better follow physics with code-built scene drafts
Video-Coder improves video generation by having an AI agent write and execute code to build a 3D scene before rendering the final video.
University of Central Florida, University of Würzburg · Nov 2026
- Top Universitycs.LG
Language models can change behavior without stronger training exposure signals
Training membership signals in LLMs arise from how information is read out, not just what is internally encoded.
Rutgers University, University of Bristol · Nov 2026
- Chinese Techcs.LG
Separating search and answer credit improves accuracy and evidence grounding
Separating search and answer credit improves both exact-match accuracy and evidence grounding in reinforcement-trained multi-turn search agents.
South China University of Technology, FiT, Tencent · Oct 2026
- Big Techcs.SD
Post-training gains in audio models may reflect format adaptation, not true audio understanding.
Post-training gains in audio-language models come from exploiting text cues and question formats, not genuine audio understanding.
National Taiwan University, NVIDIA · Oct 2026
- Academiccs.LG
LLM defenses against prompt injection are channel-conditioned and require deployment-aware training
Indirect prompt-injection defenses fail across presentation channels, while channel-aware adversarial training sharply reduces attacks without substantially sacrificing clean accuracy.
Mohamed bin Zayed University of Artificial Intelligence · Oct 2026
- Top Universitycs.CLcode
LLMs fail multi-step math despite solving each step correctly
LLMs often solve all intermediate math milestones yet fail to combine them into a correct final solution, making composition their largest diagnosed reasoning gap.
Rutgers University, Harvard University · Oct 2026
- Academiccs.LG
Pass@k misses output diversity changes caused by model post-training
pass@k measures correct-answer probability but not output diversity, so it cannot detect post-training's effects on capability breadth.
Accenture Strategy & Consulting, Vizuara AI Labs · Oct 2026
- Big Techcs.AI
Coevolving specialized models and LLM agents for autonomous self-improvement
Heterogeneous Agentic Coevolution jointly evolves specialized foundation models and LLM agents through reciprocal learning to improve domain-specific performance.
Meta AI, University of Virginia · Oct 2026
- Industrycs.AI
Code execution exposes oversight failures hidden by aggregate benchmarks
Cheap learned overseers miss rare shortcut-driven errors even when aggregate performance appears strong.
LawZero · Oct 2026
- Top Universitycs.LG
Outcome-only reinforcement learning can teach models to judge reasoning steps
Outcome-only RL can induce step-level process supervision in generative reward models, reaching 85.2 F1 on ProcessBench with 3.2K outcome-labeled trajectories.
Renmin University of China, Tsinghua University · Oct 2026
- Big Techcs.CL
Training SQL agents inside their execution harness improves accuracy
Training SQL agents inside their execution harness via SFT and RL substantially improves execution accuracy on interactive database benchmarks.
Hong Kong University of Science and Technology, Microsoft Research · Oct 2026
- Big Techcs.CL
Low-rank gradient sketches cut memory costs in language-model reinforcement learning
LoGRA reduces LLM reinforcement learning memory by up to 45.7% using low-rank gradient sketches and KL step control.
NVIDIA · Oct 2026
- Academiccs.AI
Bilevel optimization bridges the retriever-policy credit gap in agentic RL
Training retrieval before policy optimization, then co-adapting both with bilevel optimization, improves retrieval-augmented LLM agents’ question-answering accuracy while reducing retriever-update memory.
Cornell University, Institute of Science Tokyo · Oct 2026
- Big Techcs.AI
Shared scene graphs connect interaction understanding with grounded robot planning
ChronoGraph links actions on affordance parts to semantic and geometric state changes, enabling vision-language models to understand and plan 4D interactions.
ETH Zurich, Technical University of Munich · Oct 2026
- Big Techcs.AI
Training models on runtime program-state reasoning boosts automated software engineering
Training LLMs to reason about runtime program states improves repository-level patch generation, test generation, and security PoC generation.
University of California, Santa Barbara, Microsoft · Oct 2026
- Big Techcs.CV
Active audio-visual tool use improves reasoning across long recordings
OmniSeek improves long-context audio-visual reasoning by letting models iteratively retrieve and combine targeted raw audio and video evidence.
Adobe Research, University of California Davis · Oct 2026
- Chinese Techcs.CV
Forensic tool use improves explainable zero-shot image forgery detection
ATAR uses 22 forensic tools to detect and explain image forgeries through multi-turn reasoning.
CCDS, Nanyang Technological University · Oct 2026
- Big Techcs.CL
Training a concept generator improves LLM reasoning over repeated sampling.
Training a small concept generator with reinforcement learning nearly doubles a larger frozen model's reasoning pass@k on hard math problems.
Meta FAIR, Université Paris-Saclay · Sept 2026
- Big Techcs.CV
A unified evaluator scores images and pinpoints defects with explanations
VIEScore2 jointly predicts image quality scores and spatially grounded defect locations, outperforming general-purpose VLMs on both tasks.
University of Waterloo, NVIDIA · Oct 2026
- Top Universitycs.AI
Language models can manage their own context more efficiently
Giving language models direct control over editable context improves long-horizon task performance while reducing inference compute compared with externally managed context.
University of Washington, Meta Superintelligence Labs · Sept 2026
- Chinese Techcode
Reinforcement learning improves tool selection across large repositories
ToolSearcher uses reinforcement learning to optimize multi-turn tool selection at scale, outperforming existing methods.
Zhejiang University, Ant Group · Sept 2026
- Chinese Techcs.LG
Verified intermediate progress improves reinforcement learning for tool-using agents
ProCredit assigns credit to turns based on verified progress checks, improving agent training over outcome-only rewards.
University of Chinese Academy of Sciences, Institute of Neuroscience, Chinese Academy of Sciences · Sept 2026
- Chinese Techcs.LG
Differentiable confidence readouts improve reasoning accuracy and calibration together
CREDO improves reasoning accuracy and confidence calibration by replacing sampled confidence text with a deterministic, differentiable token-probability readout.
University of Science and Technology of China, Qwen Business Unit of Alibaba · Sept 2026
- Chinese Techcs.CV
One OCR model family handles recognition, parsing, and reasoning
PolyOCR achieves state-of-the-art performance across diverse OCR tasks by unifying instruction-following, a data engine, and competence-guided policy optimization.
Ant Group · Oct 2026
- Big Techcs.CR
Training recipe boosts security in AI-generated code by improving planning and testing
SecureVibe improves code security in vibe coding by training agents to plan and test for security, boosting security pass@1 by 6.9 points.
Carnegie Mellon University, Microsoft Research · Oct 2026
- Research Labcs.CL
Companion model sharpens LLM confidence without harming task performance
CoCal trains a lightweight companion on rollout hidden states to calibrate LLM confidence without sacrificing task performance.
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · Sept 2026
- Research Labcs.LG
HiLoRe allocates GRPO activations between storage, compression, and recomputation using update exposure.
HiLoRe accelerates GRPO training by allocating activations among precise storage, low-precision compression, and recomputation according to their current update sensitivity.
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Tianjin Normal University · Sept 2026
- Independentcs.AI
Composed service environments improve agents across diverse tool-use benchmarks
CompoWorld generates cross-service tasks from reusable services to train agents that outperform frontier models.
Sept 2026
- Big Techcs.CL
Eight-stage post-training recipe improves an open 106-billion-parameter language model
A reproducible eight-stage post-training recipe improves the GLM-4.5-Air-Base model beyond its official release.
Amazon · Sept 2026
- Big Techcs.AI
Post-training can replace prompt-time schema injection for enterprise coding agents
CRAFT uses two-stage post-training to embed schema knowledge into coding agents, improving analytics performance while reducing prompt token costs by 9x.
Amazon Advertising Foundations, Amazon Web Services Agentic AI · June 2026
- Top Universitycs.CLcode
Post-training amplifies a shared circuit for following misleading cues
Multiple cue-following biases in language models share one attention circuit that exists after pretraining and is activated by assistant tuning and strengthened by RL.
University of Michigan, Jinesis Lab, University of Toronto & Vector Institute · Oct 2026
- Academiccs.LG
AI agents repeat targeted mistakes about half as often after retraining
MiRL improves terminal agents by training from recurring failure points and rewarding rollouts that both pass tests and avoid the targeted misbehavior.
University of North Carolina at Chapel Hill · Oct 2026
- Chinese Techcs.LG
Language models reason better when conflicting teacher guidance is filtered
GIFT filters token-level distillation updates by their geometric alignment with RL gradients and anneals coverage to improve reasoning.
Shanghai Jiao Tong University, Tencent · Oct 2026
- Chinese Techcs.AI
Grounding rewards must check source independence and conflict handling in search agents
A hierarchical reward for source independence and conflict handling improves search-agent evidence grounding beyond outcome and citation checks.
Peking University, Tencent · Oct 2026
- Independentcs.AI
A closed-loop system lets harness agents improve without external teachers
ClawLoop is a teacher-free closed-loop framework that lets harness-based LLM agents iteratively improve by synthesizing their own training tasks.
Oct 2026
- Research Labcs.RO
Hierarchical skill-based framework achieves high precision in robotic surgery by separating semantic decisions from low-level control
SurgiCLAW confines a multimodal language model to choosing atomic surgical skills while deterministic controllers execute them, achieving 94.6% accuracy and 88.0% real-robot success.
Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
- Big Techcs.AI
GRPO Can Reward Lucky Guesses as If They Were Reasoning
GRPO's advantage estimator rewards guessed correct answers, and SIGNBALANCE fixes this by decoupling magnitude from group statistics.
Rochester Institute of Technology, Adobe Research · Sept 2026
- Academiccs.AI
Joint training helps small language models create and use tools
SMITH jointly optimizes tool creation and use via RL, achieving 79.8% accuracy on held-out reasoning tasks.
Appier AI Research, National Taiwan University · Aug 2026
- Chinese Techcs.CV
AI models can improve image understanding and generation from their outputs
Deciding per sample whether text or image is the more reliable half of a self-generated multimodal trajectory lets a unified model improve both understanding and generation without outside supervision.
Xi’an Jiaotong University, Qwen DianJin Team, Alibaba Cloud Computing · Nov 2026
- Big Techcs.CL
AI agents nearly clear proof benchmarks but harder code challenges remain
EasyProver proves almost all existing formal math and code benchmarks and introduces a harder benchmark from competitive programming.
Case Western Reserve University, Amazon Web Services · Oct 2026
- Research Labcs.LG
Self-teaching language models improve when the teacher gets feedback too
Teacher-Evolution Self-Distillation (TESD) enhances on-policy self-distillation by improving self-teacher supervision, outperforming existing methods.
Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
- Academiccs.LG
Lean proof assistant generates step-level labels to train natural-language proof verifiers.
Lean-generated step labels train better natural-language proof verifiers than human labels, without human annotation.
University of Texas at Austin, University of California, Los Angeles · Oct 2026
- Chinese Techcs.LG
Divergence geometry and estimation jointly improve language-model policy optimization
GPPO unifies LLM RL algorithms via divergence and estimator choices, outperforming standard GRPO by up to 50% on math and 14% on coding.
Institute for AI Industry Research (AIR), Tsinghua University, SIA-Lab of Tsinghua AIR and ByteDance Seed · Oct 2026
- Independentcs.LG
RL training runs can yield stronger policies than their best checkpoints
An RL training history can yield policies stronger than its best checkpoint without extra training or inference compute.
Oct 2026
- Big Techcs.LG
Token-level rewards make critic-free reasoning training faster and more accurate
BoT-GRPO uses length-normalized token rewards to train reasoning models faster than GRPO, while improving React compilation, visual quality, and AIME results without a critic.
Amazon AGI, Virginia Tech · Oct 2026
- Chinese Techcs.LG
Distilled task updates can merge better than stronger teacher updates
On-policy distillation task vectors can complement RL teacher updates and compose more effectively across tasks.
Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
- Independentcs.CL
Random rewards reveal three stages of LLM capability development
Random-reward reinforcement learning probes dormant LLM capability by revealing improvements reachable without correctness feedback, including differences hidden by equal baseline accuracy.
Oct 2026
- Chinese Techcs.CV
Spatial rewards and staged training improve image layout accuracy
An agentic reward model and curriculum learning together improve spatial accuracy in text-to-image generation.
Southeast University, Tencent HY · Oct 2026
- Big Techcs.LG
Models Learn to Allocate Context and Scale Inference More Effectively
A model's ability to choose how to allocate and reuse contexts during inference can be learned, enabling test-time scaling.
Carnegie Mellon University, AWS AI Labs · Oct 2026
- Industrycs.CV
Visual image rearrangement helps models when exact evidence matters most
Visual re-representation delivers the strongest benefits when multi-image reasoning depends on precise comparisons, transformations, orientations, or spatial evidence.
LMU Munich, MCML · Oct 2026
- Industrycs.LG
Difficulty-gated teacher guidance improves solution coverage in on-policy training
DiffGate applies bounded teacher guidance to failed trajectories only, improving pass@8 over GRPO across Qwen3 students in code and math.
Indian Institute of Science, Snap Inc. · Oct 2026
- AI Startupcs.LG
Lexicographic distillation preserves priority-ordered capabilities in multi-objective RL
LMOPD preserves top-priority capabilities by routing each rollout to the specialist for its highest unmet objective and projecting corrections.
Cohere, Mila, Université de Montréal · Oct 2026
- Big Techcs.CL
Clinical language models diagnose well but often mishandle full encounters
KlinikeBench reveals that top language models succeed on less than 30% of interactive clinical tasks despite 90.7% diagnostic accuracy.
Zhejiang University, Imperial College London · Oct 2026
- Research Labcs.LG
Muon optimizer boosts training success for long-horizon repository agents
Muon optimizer combined with token-averaged loss and auxiliary penalties improves training success in long-horizon RL for repository-level issue resolution.
Renmin University of China, Duke University · Oct 2026
- Chinese Techcs.CL
Graded rewards improve language models’ control over scoped instructions
ScopeIF uses graded rewards and a unified constraint schema to improve LLMs' precise instruction following on scope-aware constraints.
Tsinghua University, Zhipu AI · Oct 2026
- Chinese Techcs.CVcode
Motion-space analysis exposes object movement flaws in generated videos
MotionInsight diagnoses object motion deficiencies in generated videos by scoring object consistency, motion continuity, and physical plausibility from motion-space representations.
MMLab, CUHK, ByteDance Inc. · Oct 2026
- Top Universitycs.LG
Sparse verifier feedback makes broad credit assignment outperform turn targeting
Uniform reward spreading outperforms binary and targeted credit in multi-turn agent RL; coverage, not localization, is key.
Institute of Science Tokyo, Zhejiang University · Sept 2026
- Industrycs.CL
Self-evolving loop synthesizes high-quality multimodal training data
VISA uses a self-evolving loop to synthesize multimodal instruction data, improving quality via verification feedback and target model errors.
vivo AI Lab · Aug 2026
- Chinese Techcs.LG
Outcome-guided self-distillation improves LLM reasoning by adapting supervision
Outcome-guided divergence selection and prefix truncation improve on-policy self-distillation for LLM reasoning, consistently beating vanilla OPSD across model scales.
Southeast University, Huawei Noah’s Ark Lab · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.