on-policy distillation
On-policy distillation is a way to teach a smaller model by having it practice tasks on its own, while a larger teacher model watches and points out mistakes. Unlike traditional training that learns from fixed examples, here the student’s own attempts are used to decide where the teacher’s guidance is most helpful. This gives dense, step-by-step feedback without needing handcrafted rewards.
Across the papers, researchers find that careful supervision design matters more than just using a strong teacher. Improvements come from when to distill (before reinforcement learning), which tokens to emphasize (constraint-aware or decisive conflicts), how to handle multiple teachers (token-level routing), and adapting distillation to multi-turn agents or teams. The common thread is that successful on-policy distillation requires aligning teacher signals with the student’s current capabilities and future impact, rather than blindly copying teacher outputs.
- Papers
- 131
- Released code
- 4
- First seen
- Sept 2026
- Latest
- Nov 2026
130 papers in the last two months, against 1 in the two before.
Start here
Three papers to read first, in the order they came out- 1AI agents can recover forgotten skills from earlier training snapshots
Agentic-OPD recovers lost capabilities from earlier RL checkpoints via on-policy distillation, improving agent performance beyond the best checkpoint.
Nov 2026 - 2Language models can inherit hidden backdoors from their AI teachers
On-policy distillation efficiently transfers conditional backdoors from teacher to student models with minimal trigger exposure.
Nov 2026 - 3Language models can beat answer-key training by solving easier problems
Label-free self-distillation outperforms voting by using easier problems the model solves consistently as teacher context.
Nov 2026
How fast it is moving
Often studied with
privileged informationInformation available only during training that improves performance but is not accessible at inference.34 togetherreinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.23 togethercredit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.20 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.LG
Language models can inherit hidden backdoors from their AI teachers
On-policy distillation efficiently transfers conditional backdoors from teacher to student models with minimal trigger exposure.
Alibaba Group, Wuhan University of Technology · Nov 2026
- Chinese Techcs.LG
AI agents can recover forgotten skills from earlier training snapshots
Agentic-OPD recovers lost capabilities from earlier RL checkpoints via on-policy distillation, improving agent performance beyond the best checkpoint.
Baidu, Inc., Beijing Jiaotong University · Nov 2026
- Big Techcs.LG
Language models can beat answer-key training by solving easier problems
Label-free self-distillation outperforms voting by using easier problems the model solves consistently as teacher context.
Amazon, Worcester Polytechnic Institute · Nov 2026
- Top Universitycs.LG
Merging specialist AI models saves 42% of training compute compared to joint training
On-policy distillation beats GRPO in compute efficiency only when using the student's own GRPO checkpoint as teacher.
Salesforce AI Research, UIUC · Nov 2026
- Academiccs.CV
Video reasoning models improve by auditing their own use of visual clues
A model can audit its own visual reliance and focus training on under-utilized visual tokens, improving video reasoning with far fewer resources.
University of Texas at Austin · Nov 2026
- Chinese Techcs.AI
Language model and skill set co-evolve through response feedback
SkillDuet coordinates model parameter updates and skill retention using response feedback, achieving 98.57% on ALFWorld and 92.97% on WebShop.
Alibaba · Nov 2026
- Chinese Techcs.LG
Language models improve at math when teachers steer their practice
Coupling student and teacher rollouts with a rejection-sampling rule raises math benchmark accuracy above on-policy distillation.
Tsinghua University, Harbin Institute of Technology, Shenzhen, China · Nov 2026
- Chinese Techcs.CV
Code simulations teach video generators realistic physics with little data
Executable code as privileged context in on-policy self-distillation dramatically improves physical plausibility in video generation.
HKUST, Alibaba Group · Nov 2026
- Big Techcs.LG
Decoupling direction and distance improves on-policy distillation for language models
D^2-OPD decouples extrapolation direction from distance in on-policy distillation, outperforming baselines by up to 9.45 points on math reasoning.
Columbia University, AWS AI Lab · Nov 2026
- Chinese Techcs.LG
Distilling AI reasoning works better when teacher rewards student's own learned math steps, not style
Student-Delta On-Policy Distillation improves math reasoning by using a student's own post-training changes to guide teacher feedback.
Zhejiang University, State Lab of CAD & CG · Nov 2026
- Chinese Techcs.AI
Parallel training method speeds language model inference by 3.5x
SpecOPD decouples on-policy supervision from sequential rollouts, enabling parallel distillation for speculative decoding.
Alibaba, University of Science and Technology of China · Nov 2026
- Research Labcs.LG
Distillation under-weights the tokens a model is most confident about
Standard on-policy distillation starves high-confidence tokens of gradient; matching teacher and student probabilities directly fixes that and improves reasoning scores.
University of Surrey, University College London · Nov 2026
- Chinese Techcs.CL
Language models can lose math skills from more math training
On-Policy Mid-Training Distillation improves language model math performance by training on student-generated trajectories instead of fixed prefixes.
Fudan University, The Hong Kong Polytechnic University · Nov 2026
- Chinese Techcs.CL
Language model distillation improves when teachers guide only the first few tokens
Aligning student to teacher on only the first 20% of generated tokens, with overlap-gated divergence, outperforms top-k on-policy distillation.
Renmin University of China, Huawei Technologies Ltd. · Nov 2026
- Top Universitycs.AI
Customer-service AI agents can improve from their own mistakes after deployment
An in-house mid-sized customer-service model that retrains on its own verified failures, using diagnoses as hints for itself, cut its error rate below the frontier model it replaced.
Meituan, University of Melbourne · Nov 2026
- Big Techcs.CL
Language models improve using self-generated questions and answers, without human examples
Large language models self-improve using only self-generated questions and privileged reasoning, without human-curated data.
Google, Ulm University · Oct 2026
- Big Techcs.AI
Language models can improve by learning from their weakened copies
Contrasting a model with its weakened copy improves reasoning accuracy without privileged information.
Massachusetts Institute of Technology, NVIDIA · Oct 2026
- Big Techcs.CV
AI models struggle to follow events across directions in panoramic video
Panoramic video reasoning models can improve by 12.5 points through sphere-time grounded reasoning training.
Nanjing University, NVIDIA · Nov 2026
- Research Labcs.LG
Safety training can make chatbots refuse even after they start complying
Adversarial Self-Distillation lets a language model recover from accidentally starting a compliant response, improving jailbreak resistance.
University of Cologne, CISPA Helmholtz Center for Information Security · Oct 2026
- Chinese Techcs.CL
Teacher rewrites of drifting student reasoning stabilize training and improve accuracy
Adaptive teacher intervention and supervision selection stabilize long-sequence on-policy distillation and improve student accuracy.
Huawei Technologies, Harbin Institute of Technology · Oct 2026
- Big Techcs.LG
Distillation overshoot lets smaller agents surpass teachers, and a new method preserves those gains.
Success-preserving on-policy distillation prevents teacher supervision from erasing effective student behaviors, yielding agents that outperform ordinary distillation and their teachers.
University of Southern California, University of Florida · Oct 2026
- Chinese Techcs.LG
AI assistants watching live video get better at timing their responses
A streaming video assistant trained by distilling its own outputs with privileged hints responds at the right moment with better content.
Qwen Business Unit of Alibaba · Oct 2026
- Top Universitycs.LG
Coding agents can learn neglected behaviors without sacrificing baseline performance
Combining per-turn textual feedback self-distillation with RL stabilizes training and improves hard-to-optimize agent behaviors.
Cursor Research, UC Berkeley · Nov 2026
- Big Techcs.CL
Teacher rollouts repair myopic token scoring in agent distillation
Teacher next-token scores are myopic; repairing them with takeover-informed scores before distillation improves agentic on-policy distillation.
NVIDIA · Oct 2026
- Big Techcs.LG
Agents improve by distilling hindsight from their own interactions
Retrospective Self-Distillation lets agents evolve from their own trajectories using hindsight and cross-task precedents without external supervision.
Meta AI · Oct 2026
- Chinese Techcs.LG
Sequence-level credit assignment improves on-policy distillation of reasoning models
GR-OPD outperforms standard on-policy distillation by assigning sequence-level credit and calibrating with group-relative preferences.
Shanghai Jiao Tong University, Alibaba Group · Oct 2026
- Research Labcs.LG
Self-teaching language models improve when the teacher gets feedback too
Teacher-Evolution Self-Distillation (TESD) enhances on-policy self-distillation by improving self-teacher supervision, outperforming existing methods.
Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
- Chinese Techcs.CV
MLLMs miss text readable by squinting; diagnosis and training method recovers it
Multimodal LLMs fail to read squint text because glyph identity is weakly encoded; training with downscaled views and transcriptions lifts ANLS from 26.50 to 88.61.
Beijing University of Posts and Telecommunications, Tencent · Oct 2026
- Big Techcs.CL
Refining user representations improves the fidelity of AI user simulation
User representations optimized via hindsight distillation enable more faithful simulation than representations from GPT-5.5.
University of Illinois Urbana-Champaign, Microsoft Research · Oct 2026
- Big Techcs.LG
Reward-guided token weighting makes language-model distillation more effective
Reward-Aligned Distillation weights teacher tokens by predicted effect on student reward and improves math and code reasoning at matched training budgets.
UC San Diego, Purdue University · Oct 2026
- Chinese Techcs.LG
Projecting conflicting gradients during joint RL and distillation improves agent training
CORD improves joint RL and self-distillation by projecting conflicting gradients to preserve a protected objective.
The Chinese University of Hong Kong, Tencent · Oct 2026
- Big Techcs.LG
Teacher training determines whether on-policy distillation preserves prior language skills
RLVR-trained distillation teachers preserve prior capabilities better than SFT-trained teachers, while out-of-domain predictive change tracks forgetting most consistently.
Microsoft Research, Nankai University · Oct 2026
- Big Techcs.CL
Training on self-generated answers improves reasoning in low-bit language models
OnQ recovers reasoning in low-bit LLMs by training quantized students on their own generated responses with candidate-wise Dynamic KL, outperforming fixed-response distillation.
University of Michigan, Ann Arbor, University of California, Santa Barbara · Oct 2026
- Big Techcs.LG
Adaptive distillation triggers teacher takeover when student diverges
Divergence-triggered teacher takeover during on-policy distillation stabilizes long-horizon training and accelerates learning from base initializations.
Google, University of Washington · Oct 2026
- Big Techcs.RO
Humanoid repositions bulky objects using its arms and torso
A humanoid robot can reposition objects using its entire body, not just hands, after training in simulation.
Amazon, University of California, San Diego · Oct 2026
- Academiccs.LG
On-policy distillation fails to improve models on problems they already get wrong
On-policy distillation degrades accuracy when training is limited to problems a student currently fails, despite decreasing KL divergence; it helps when correct prefixes dominate.
Technical University of Munich, National Institute of Informatics · Oct 2026
- Chinese Techcs.CV
Agentic image checks test depicted facts, not AI provenance
EviScene is an agent that assesses image factuality by iteratively refining questions and gathering external evidence.
Jilin University, Institute of Automation, Chinese Academy of Sciences · Oct 2026
- Chinese Techcs.LG
Grounding distillation in interaction states improves multi-turn agent performance
In multi-turn agent distillation, grounding supervision in interaction states and resampling complete reasoning-action responses improves task success and action diversity.
Alibaba Group, Tsinghua University · Oct 2026
- Chinese Techcs.CV
Decoupling skills from expert policies improves multi-capability vision-language model training
SCOPED trains capability-specific experts and distills explicit skills into a unified VLM, outperforming mixed-capability RL.
Northeastern University, Alibaba Group · Oct 2026
- Chinese Techcs.AI
Shaping teacher behavior improves self-distillation for language models
TeachBack uses bidirectional preference optimization to shape teacher trajectories for improved student reasoning in self-distillation.
Huazhong University of Science and Technology, Tsinghua University · Oct 2026
- Big Techcs.LG
Token-level teacher selection improves on-policy distillation for LLMs
Token-level teacher selection in on-policy distillation improves student capability and diversity by over 10 percentage points on several tasks.
Princeton University, Amazon Annapurna Labs · Oct 2026
- Research Labcs.AI
Offline trajectories can train agents without environment interaction while limiting forgetting
SAPD improves LLM agent training by distilling offline trajectories with relative step advantages, requiring no environment interaction.
Meituan, University of Chinese Academy of Sciences · Oct 2026
- Chinese Techcs.AI
Branching student trajectories improves on-policy distillation efficiency and accuracy
Branching student trajectories at uncertain decisions improves on-policy distillation accuracy on math and agent benchmarks while generating fewer tokens than independent rollouts.
Tsinghua University, SenseTime Research · Oct 2026
- Chinese Techcs.AI
Condition-aware distillation reduces invalid reasoning in smaller language models
Condition-contrastive distillation, which compares teacher scores with and without a question condition, reduces invalid continuations and improves distilled LM reasoning.
The Chinese University of Hong Kong, Shenzhen, Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen) · Oct 2026
- Top Universitycs.LG
Post-training order determines how well language models learn next
In LLM post-training, a stage that improves the current model can make the next stage less effective.
Salesforce AI Research, UIUC · Oct 2026
- Independentcs.CL
Offline student rollouts often beat on-policy distillation for language models
Semi-OPD, which uses initial offline student rollouts, often outperforms on-policy distillation across diverse teacher-student pairs.
Oct 2026
- Chinese Techcs.CL
Supervision reliability trumps alignment coverage in cross-tokenizer distillation
In cross-tokenizer on-policy distillation, expanding alignment coverage with span supervision reduces accuracy; compact strict supervision is more effective.
Alibaba Cloud Computing · Oct 2026
- Chinese Techcs.CV
Refined Questions and Teacher Guidance Improve Long-Horizon Multimodal Search
MMSearch-OPD improves long-horizon multimodal search agents by combining data refinement with joint reinforcement learning and on-policy distillation.
Peking University, HKUST(GZ) · Oct 2026
- Chinese Techcs.LG
A model can become its own stronger teacher by looping selected layers
LoOPSD uses repeated layers as a teacher to improve student model reasoning through on-policy self-distillation.
Anhui Province Key Laboratory of Digital Security, Token Foundry, Alibaba Group · Oct 2026
- Big Techcs.LG
Safety distillation can pass hidden backdoors from teacher to student
On-policy distillation for safety can propagate teacher backdoors to students at low poisoning rates.
Stony Brook University, Amazon · Oct 2026
- AI Startupcs.LG
Lexicographic distillation preserves priority-ordered capabilities in multi-objective RL
LMOPD preserves top-priority capabilities by routing each rollout to the specialist for its highest unmet objective and projecting corrections.
Cohere, Mila, Université de Montréal · Oct 2026
- Chinese Techcs.LG
Distilled task updates can merge better than stronger teacher updates
On-policy distillation task vectors can complement RL teacher updates and compose more effectively across tasks.
Institute of Automation, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
- Academiccs.AI
Open dataset and distillation-to-reinforcement pipeline sharpen GUI grounding in 8B model
A large GUI grounding dataset and a distillation-to-reinforcement learning training method yield state-of-the-art 8B-scale grounding models.
Prentis AI, Westlake University · Oct 2026
- Chinese Techcs.LG
Uncertainty-weighted credit improves cross-tokenizer distillation for LLMs
CAST improves cross-tokenizer distillation by assigning credit per student token using uncertainty and local teacher smoothness.
Taobao & Tmall Group of Alibaba, Singapore Management University · Oct 2026
- Big Techcs.CV
Self-distillation helps vision-language models reason with far fewer visual tokens
Visual token pruning degrades VLM performance primarily due to unreliable use of remaining visual evidence, not irreversible information loss.
University of Toronto, Vector Institute · Oct 2026
- Chinese Techcs.AI
Models confuse rejected proposals with active instructions in dialogue
Models treat mentioned but rejected changes as still active, degrading accuracy in multi-turn tasks.
HKUST, Tencent Hy · Oct 2026
- Big Techcs.LG
Joint on-policy learning improves self-distillation by aligning teacher with student
A privileged teacher improves RL students only if its distillation update aligns with the student's reward gradient, motivating joint on-policy training.
Perplexity, NVIDIA · Oct 2026
- Chinese Techcs.CL
Specialized memory writer improves persistent state across multiple LLM frameworks
A specialized memory writer trained with attributed chunk-level reinforcement learning outperforms general-purpose models across four long-term-memory harnesses.
Tencent Hunyuan · Oct 2026
- Independentcs.CV
Post-training recipe makes compact multimodal reasoners beat larger models
MMVistaReason demonstrates that systematic open-data construction and capacity-aware post-training provides a practical path to reliable multimodal reasoning.
Oct 2026
- Big Techcs.AI
Training agents around decisive errors improves prevention and recovery
PivotOPD jointly trains agents to prevent and recover from pivotal mistakes, achieving top average performance across three benchmarks.
Princeton University, NVIDIA · Oct 2026
- Independentcs.CL
On-policy distillation can transfer think-mode advantage to no-think responses via trace-response routing
ThinkOPD outperforms uniform on-policy distillation by combining reward gain and trace-response divergence for routing.
Oct 2026
- Chinese Techcs.CLcode
Adaptive tutoring helps rerankers assemble better evidence sets for research
AdaTutoRank, a setwise reranker with adaptive tutoring hints matched to rollout quality, tops ten RAG and deep research benchmarks while using fewer retrieval calls.
University of Science and Technology of China, Yuanbao Team, Tencent · Sept 2026
- Chinese Techcs.CL
Adaptive teacher intervention boosts distillation accuracy and stability
Adaptive teacher intervention guided by policy disagreement improves accuracy and reduces training length in on-policy distillation.
Nanyang Technological University, Baidu Inc. · Oct 2026
- Big Tech
Privileged self-practice beats self-distillation for multi-turn language-model agents
Privileged Self-Practice injects task-specific instructions into failing agent rollouts to improve multi-turn agent training.
Texas A&M University, AWS AI, Amazon · Sept 2026
- Chinese Techcs.CL
Teacher-guided training improves specialization and coordination in multi-agent models
MAS-OPD achieves the highest mean score on code and math benchmarks by using on-policy distillation with role-advantage specialization and privileged attribution.
University of Science and Technology of China, Tencent · Sept 2026
- Chinese Techcs.LG
Adaptive per-trajectory stopping speeds on-policy distillation by up to 7.5x
Flash-OPD speeds up on-policy distillation by 2.2 to 7.5 times while maintaining or improving accuracy.
Tencent Hy, HKUST(GZ) · Oct 2026
- Big Techcs.LG
Training the teacher on student prefixes boosts on-policy distillation performance
Adapting the teacher to student-generated prefixes improves on-policy distillation.
Washington University in St. Louis, AWS AI Labs · Oct 2026
- Chinese Techcs.CL
Hierarchical supervision allocation improves long-horizon model distillation
Hierarchical supervision allocation via Locate-Validate-Refine improves long-horizon on-policy distillation for LLM agents.
Ant Group, Alibaba International Digital Commerce Group · Sept 2026
- Chinese Techcs.AI
Segment-wise distillation improves LLM reasoning by learning to revise intermediate steps.
Segment-wise On-Policy Distillation improves reasoning by training students to prefer whole teacher redrafts of their reasoning segments.
The Hong Kong University of Science and Technology (Guangzhou), University of Science and Technology of China · Oct 2026
- Industrycs.LG
Difficulty-gated teacher guidance improves solution coverage in on-policy training
DiffGate applies bounded teacher guidance to failed trajectories only, improving pass@8 over GRPO across Qwen3 students in code and math.
Indian Institute of Science, Snap Inc. · Oct 2026
- Chinese Techcs.LGcode
Token-level teacher routing beats prompt-level routing in multi-teacher distillation
Token-level ExpertAlign routing combines specialized teachers without domain labels and outperforms prompt-level routing and mean aggregation across four distillation settings.
Shanghai Jiao Tong University, GAIR · Sept 2026
- Chinese Techcs.LG
Models can absorb spatial reasoning from verified agent traces
SpatialOPSD uses on-policy self-distillation from verified coding agent traces to give MLLMs tool-free spatial reasoning, beating SFT and GRPO.
Alibaba Group · Oct 2026
- Chinese Techcs.LG
Agent distillation improves when supervision follows outcomes, not model disagreement
Outcome-calibrated teacher supervision improves multi-turn agent distillation because token-level teacher-student gaps alone do not predict which interventions improve task success.
Qwen Large Model Application Team, Alibaba, Peking University · Sept 2026
- Big Techcs.AI
General instructions outperform gold answers in on-policy context distillation
Using general instruction privileges instead of instance-specific gold improves out-of-distribution performance in On-Policy Context Distillation.
Columbia University, Amazon Web Services · Oct 2026
- Chinese Techcs.LG
Execution graph helps off-the-shelf teachers supervise student agents better
Graph-conditioned retrieval of teacher execution histories improves on-policy distillation for compact language agents in multi-turn tasks.
Yuanbao Team, Tencent, Tsinghua University · Sept 2026
- Big Techcs.LGcode
Future-aware distillation corrects teacher-student information mismatch in block diffusion language models
Correcting an autoregressive teacher's next-token distribution with visible future context improves distillation of block diffusion language models from pretrained AR models.
Tsinghua University, MIT · Oct 2026
- Chinese Techcs.LG
Ablating individual constraints improves distillation for complex instruction following
CC-OPD improves multi-constraint instruction following by using per-constraint probability differentials as token-level rewards.
Alibaba Group · Sept 2026
- Big Techcs.LG
Attention distillation complements token-level supervision for reasoning models
On-policy attention self-distillation (OPASD) improves reasoning accuracy by 4.98 to 8.40 percentage points while reducing training compute by 72.6% compared to token-only distillation.
ACI PLC, University of Dhaka · Sept 2026
- Big Techcs.CV
Last-frame layouts guide video generation through large camera moves
LIFT enables specifying layout of future views in video generation via on-policy self-distillation from a dense-layout teacher, improving controllability under large viewpoint changes.
UC San Diego, University of Virginia · Oct 2026
- Big Techcs.CL
Selective self-distillation helps LLM judges generalize on subjective evaluations
Training LLM judges with self-distillation and masking high entropy shift positions improves subjective task accuracy by 2 to 9 percentage points.
Georgia Institute of Technology, Amazon · Oct 2026
- Big Tech
Co-evolving teacher and student models improves mathematical reasoning
A recursive distillation method with a co-evolving privileged teacher improves reasoning accuracy more than using a frozen teacher.
Meta AI, University of California, Riverside · Sept 2026
- Chinese Techcs.LG
Large-to-small diffusion model distillation fails due to CFG error amplification, new method resolves it
Folding a large diffusion teacher’s guided policy into a small student’s conditional branch prevents guidance error amplification and improves cross-scale distillation.
Hefei University of Technology, Tsinghua University · Oct 2026
- Chinese Techcs.CV
Real-time GUI feedback improves training for computer-use agents
ComputerSD uses real-time GUI feedback to improve computer-use agent training via online self-distillation.
Zhejiang University, Ant Group · Oct 2026
- Chinese Techcs.AI
Self-distillation trains GUI agents for longer, memory-dependent tasks
GUI-SD-v2 extends on-policy self-distillation to multi-turn GUI agents, outperforming state-of-the-art methods.
Institute of Information Engineering, Chinese Academy of Sciences, Tencent · Sept 2026
- Chinese Techcs.LG
Training agents on their own trajectories accelerates reward-based learning
On-policy warmup using teacher supervision on student trajectories accelerates and improves agentic reinforcement learning under sparse rewards.
Zhejiang University, Ant Healthcare · Oct 2026
- Top Universitycs.CLcode
Distillation Before Reinforcement Learning Improves Reasoning Model Post-Training
A two-stage scheme of on-policy distillation followed by reinforcement learning outperforms joint optimization for reasoning LLMs.
New York University, University of Chicago · Sept 2026
- Chinese Techcs.CL
Language models learn from scoring rules that otherwise teach nothing
Inactive rubrics in reinforcement learning can be recovered by using them as teachers for missing behaviors and critics for finer discrimination.
Zhejiang University, Huawei · Nov 2026
- Chinese Techcs.CV
Image editors can drift less while retaining most existing capabilities
TailSteer reduces cumulative drift in multi-turn image editing by applying preservation corrections exclusively in the final denoising steps.
MMLab, CUHK, SenseTime Research · Nov 2026
- Big Techcs.CL
AI coding model beats predecessor using roughly one-fifth the active parameters
Nemotron 3.5 Super trains a 12B-active multimodal model that exceeds a 4.6x larger model on SWE-bench Pro.
NVIDIA · Oct 2026
- Chinese Techcs.AI
AI web agents learn long tasks by reusing skills salvaged from failures
SCOPE achieves 3.9% to 18.6% relative gains on web tasks by progressively discovering and composing local skills.
Renmin University of China, Huawei Technologies · Nov 2026
- Chinese Techcs.LG
AI assistants that learn separate style modules from response feedback outperform fixed-prompt personalization
PLF learns reusable assistant personas from feedback using separate LoRA adapters, enabling personalized responses without per-user finetuning.
UC San Diego, Alibaba Group · Oct 2026
- Big Techcs.RO
Coding agents teach humanoid robots new whole-body skills without retraining
A single frozen Metatron checkpoint supports walking, recovery, object pickup, and sitting on a real humanoid through code-defined prompts.
NVIDIA, Dollhouse Robotics · Oct 2026
- Chinese Techcs.AI
Feedback criteria that track a model's growing ability halve its training steps
Co-evolving a reward rubric alongside a policy by tracking its capability frontier improves and accelerates open-ended LLM training.
University of Science and Technology of China, Alibaba Group · Oct 2026
- Industrycs.CV
AI models can answer image questions and refine pictures they generate
A unified discrete diffusion LLM can work for both understanding and generating images, using self-refinement and cross-paradigm distillation to match specialized models.
Oct 2026
- Academiccs.AI
Realistic simulated users improve training for interactive language agents
MIMESIS learns realistic user behaviors from human conversations and reasoning traces, enabling agents to train more effectively and generalize across unseen users.
Meta Superintelligence Labs, New York University · Oct 2026
- Chinese Techcs.CL
Residual advantage uses teacher-student disagreement as bounded step reward
Residual Advantage improves RLVR by using a bounded, centered teacher-student residual as a step-level credit allocation signal.
Harbin Engineering University, Tencent · Oct 2026
- Chinese Techcs.AI
Small agents learn when to seek cloud help
Sibyl achieves 95.2% success on ALFWorld and 80.4% on WebShop, averaging 0.8 and 3.9 cloud calls per trajectory.
Hangzhou Dianzi University, Fudan University · Oct 2026
- Big Techcs.SE
New benchmark tests AI's ability to spot invalid code reviews
AI agents can judge if a code review comment is technically correct by gathering repository evidence, outperforming general LLMs.
University College London, Amazon · Sept 2026
- Chinese Techcs.CV
One OCR model family handles recognition, parsing, and reasoning
PolyOCR achieves state-of-the-art performance across diverse OCR tasks by unifying instruction-following, a data engine, and competence-guided policy optimization.
Ant Group · Oct 2026
- Chinese Techcs.AI
Selective distillation improves safety alignment while halving rollout compute and preserving reasoning.
EOPSA improves safety alignment efficiency by focusing training only on reliably supervised, safety-critical tokens.
Zhejiang University, Ant Group · Sept 2026
- Chinese Techcs.AI
PlanGuard detects physical risks across complete embodied-agent plans
PlanGuard detects physical safety risks in complete multi-step embodied plans before execution, reaching 87.15% accuracy and 87.21% F1 at 2B scale.
Anhui Province Key Laboratory of Digital Security, The University of Hong Kong · Sept 2026
- Big Techcs.LG
Minimal corrections teach hard tasks while preserving broader model skills
Minimal, refreshed corrections of failed LLM attempts improve hard-task specialization while preserving prior capabilities better than on-policy training or full-answer replacement.
AWS AI Labs, University of Massachusetts · Oct 2026
- Big Techcs.LG
Reverse-KL distillation's sample needs hinge on who generates trajectories
Online reverse-KL distillation with full teacher feedback requires fewer samples than with action-only feedback.
University of Virginia, Google Research · Oct 2026
- Academiccs.RO
One-camera humanoids learn to catch objects they have never seen
A humanoid robot can catch unseen objects with underhand and overhand throws using one camera.
UC San Diego, Harvey Mudd College · Oct 2026
- Chinese Techcs.LG
Guided self-exploration lets language models teach themselves better reasoning
SEPO uses labels to guide LLM reasoning exploration, improving self-evolution efficiency and final performance.
Renmin University of China, Peking University · Oct 2026
- Chinese Techcs.AI
Synthetic cowork environments improve agents’ handling of changing shared workspaces
Cowork-Gym’s synthesized environments improve Qwen agent models by 4.1 to 8.7 points on six benchmarks.
Alibaba Token Foundry · Oct 2026
- Chinese Techcs.AI
Localized Context Distillation resolves the learning dilemma in interaction experience
Localized Context Distillation improves LLM learning from interaction experience by having a teacher regenerate only critical spans of each student trajectory.
Tencent, The Chinese University of Hong Kong · Oct 2026
- Big Techcs.AI
Turning failed reasoning attempts into training signal improves GRPO on hard problems
Failed reasoning attempts can be turned into verified supervision for GRPO by having the policy write corrective notes and distilling only replays that beat the original group.
University of California, San Diego, Adobe Research · Oct 2026
- Big Techcs.LG
Progress-aware training helps small language model agents escape repetitive loops
ProD detects and penalizes stagnation in small language model agents, improving task success and efficiency.
The Pennsylvania State University, Microsoft CoreAI · Oct 2026
- Big Techcs.LG
Persistent evidence helps long-horizon agents recover from earlier mistakes
Persistent belief updates across rollback plus an implicit world model improve failure recovery in long-horizon agents, and distillation cuts teacher cost.
University of Maryland, Google · Oct 2026
- Chinese Techcs.CV
Benchmark and method for true audio-visual joint reasoning
OmniReasoning framework improves audio-visual joint reasoning accuracy by 12.8 and 9.3 points on two benchmarks.
Peking University, Alibaba Group · Oct 2026
- Chinese Techcs.CV
Clinical agents learn to gather evidence across longitudinal MRI records
CASE agents achieve over 16% and 10% relative improvements in answer accuracy over GPT-5.4 and Claude Opus 4.8 on longitudinal medical reasoning.
Durham University, DAMO Academy, Alibaba Group · Oct 2026
- Independentcs.LG
Agentic RL framework keeps asynchronous training stable with bounded staleness
LoongSage makes agentic LLM reinforcement learning more efficient through asynchronous execution, bounded trajectory staleness, topology-aware weight transfer, and train-inference consistency.
Oct 2026
- Chinese Techcs.CL
Recovery-aware calibration improves distillation recovery for pruned reasoning models
ReCal improves post-pruning on-policy distillation recovery by reweighting calibration statistics based on teacher-probe token-level forward KL.
Zhongguancun Academy, Tencent · Oct 2026
- Big Techcs.LG
Activation steering strengthens token-level supervision for self-distilled reasoning
Activation-Conditioned Self-Distillation steers a frozen self-teacher using contrasts from verified correct trajectories, improving math and code accuracy beyond reference-conditioned self-distillation.
Rensselaer Polytechnic Institute, IBM Research · Oct 2026
- Chinese Techcs.LG
Online distillation cuts needless verbosity in RL-trained models
Length Self-Distillation reduces excess response length on already-solved queries without sacrificing accuracy.
ByteDance Seed, Tongji University · Oct 2026
- Chinese Techcs.CL
Terminal agents verify often but miss and mishandle many errors
Agents self-verify almost always but detect only 61.43% of errors and repair only 49.36% of those; SCVD improves outcomes.
Northeastern University, Inclusion AI, Ant Group · Oct 2026
- Big Techcs.RO
HULK teaches humanoids to carry heavy loads while staying balanced
HULK uses model predictive control to guide reinforcement learning for humanoid whole-body forceful loco-manipulation under sustained payloads.
University of Michigan, University of California, Los Angeles · Oct 2026
- Top Universitycs.CL
Teacher corrections with dense feedback double AI success on hard logic problems
PIVOT nearly doubles mean avg@8 on hard logic games from 19.5% to 35.6%.
University of Chinese Academy of Sciences, Tsinghua University · Oct 2026
- Big Techcs.AI
Consistency-regularized co-evolution of knowledge and model improves LLM agents
LLM agents evolve more reliably by selecting experience-derived knowledge that yields consistent utility gains across multiple analysis views, rather than relying on any single interpretation.
University of Illinois Urbana-Champaign, Georgia Institute of Technology · Oct 2026
- Big Techcs.LG
Restricting weight updates to non-principal subspace preserves old skills while learning new ones
Subspace Tuning prevents catastrophic forgetting by restricting LLM fine-tuning updates to non principal weight subspaces.
UC Berkeley, Mila · Oct 2026
- Big Techcs.LG
Frozen LLM critics sharpen credit assignment for long-horizon agents
A frozen LLM value critic, policy-aware via in-context conditioning, improves step-level credit assignment in agentic RL, beating GRPO and PPO.
Apple · Oct 2026
- Big Techcs.RO
Minutes of supervised play bootstrap hours of autonomous robot practice
A robot learns dexterous manipulation by mimicking minutes of human play, then autonomously practices to train a world model and guide control.
Amazon Frontier AI & Robotics (FAR), University of Maryland, College Park · Oct 2026
- Chinese Techcs.LG
Blending teacher and student token distributions improves distillation of language models.
Interpolating student and teacher next-token distributions during rollout yields better distillation than using either endpoint alone.
Tencent Inc., University of Science and Technology of China · Oct 2026
- Independentcs.CV
Training long-horizon multimodal research agents with reinforcement learning at scale
Long-MDR makes 128k-context, 75-turn multimodal reinforcement learning more efficient and stable through distillation warmup, gradual horizon expansion, and entropy-based recovery.
Oct 2026
- Big Techcs.LG
Post-training multi-token prediction heads match joint pretraining with far less data
Post-trained MTP heads on a frozen model match jointly-pretrained quality with 10^3-10^4x fewer tokens; relaxed verification and adaptive head count add speedups.
Microsoft · Oct 2026
- Chinese Techcs.CV
Independent chunk scoring reduces drift in long autoregressive video rollouts
Rollout-Marginal Distillation scores each generated chunk independently against a teacher, then refines temporal coherence, preventing error accumulation in long autoregressive video generation.
MMLab, The Chinese University of Hong Kong · Oct 2026
- Chinese Tech
Dynamic optimization loop aligns 3D generators with 2D diffusion priors to improve realism
OREO enhances 3D generation fidelity by editing rendered views and distilling the gap back into the generator via latent contrastive learning.
The Hong Kong Polytechnic University, Tencent ARC Lab · Oct 2026
- Big Techcs.LG
Small language models outperform 763B model at analog circuit design
A small locally-deployed model fine-tuned with reinforcement learning outperforms a massive cloud API model on multi-turn analog circuit optimization.
MIT, Analog Devices, Inc. · Nov 2026
- Chinese Techcs.SD
Audio-language model suite adds speaker-aware transcription and understanding
Qwen-Audio-3.1-ASR improves multilingual speech recognition and adds speaker-labeled, timestamped transcription and broader audio understanding through variant-specific capabilities.
Alibaba Group · Oct 2026
- Top Universitycs.LG
Divergence choice in distillation controls student model entropy
Divergence choice in distillation controls student entropy, with forward KL inflating and reverse KL deflating it.
Stanford University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.