exploration bias
When an agent explores, it often develops a tendency to favor certain options based on
- Papers
- 21
- Released code
- 3
- First seen
- Sept 2026
- Latest
- Nov 2026
21 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Structural guidance improves language models’ long-horizon reasoning under sparse rewards
SAGE improves sparse-reward, long-horizon LLM reasoning by pruning structurally unproductive branches and supplying dense geometric guidance across reasoning depth.
Sept 2026 - 2Correcting search bias makes deep research conclusions more representative
CESS corrects evidence selection bias in deep research agents by weighting documents by their selection probabilities.
Oct 2026 - 3MaskSense preserves subject exploration when training masked image generators with RL
MaskSense uses semantic priors and entropy anchoring to avoid a visual exploration trap in reinforcement learning for masked image generation.
Oct 2026
How fast it is moving
Often studied with
reinforcement learning with verifiable rewardsReinforcement learning that uses rewards from objectively checkable signals like correct answers or verification feedback.6 togetherreinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.6 togethercredit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.5 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Academiccs.AI
Correcting search bias makes deep research conclusions more representative
CESS corrects evidence selection bias in deep research agents by weighting documents by their selection probabilities.
Beijing Normal University, Ke Holdings · Oct 2026
- Chinese Techcs.CV
MaskSense preserves subject exploration when training masked image generators with RL
MaskSense uses semantic priors and entropy anchoring to avoid a visual exploration trap in reinforcement learning for masked image generation.
University of Science and Technology of China, Tongyi Lab, Alibaba Group · Oct 2026
- Top Universitycs.AIcode
Structural guidance improves language models’ long-horizon reasoning under sparse rewards
SAGE improves sparse-reward, long-horizon LLM reasoning by pruning structurally unproductive branches and supplying dense geometric guidance across reasoning depth.
Virginia Tech, University of Wisconsin Madison · Sept 2026
- Top Universitycs.RO
Touch-directed curiosity helps robots discover useful manipulation skills
Tactile-driven curiosity lets robots learn manipulation and grasp policies without task rewards or demonstrations, and improves post-training of VLA models.
ETH Zürich, University of California, Berkeley · Oct 2026
- Big Techcs.CL
Search results order and dead-end histories bias AI agents' final answers
Final answers of search-equipped LLM agents are more sensitive to initial evidence order than when all evidence is supplied, and dead-end work reduces strategy switching.
Amazon Web Services · Nov 2026
- Big Techcs.RO
Humans intervene when robots stall, not when they err
Adding optimistic bonuses to human intervention transitions improves robot learning efficiency by 10-50%.
University of Washington, Toyota Research Institute · Nov 2026
- Big Techcs.LG
LLM optimizers often stall within narrow families of design principles
Feedback-driven LLM optimizers often collapse at the level of design principles, not candidates, causing plateaus that diversity injection fails to fix.
The University of Manchester, University of Chinese Academy of Sciences · Oct 2026
- Top Universitycs.AIcode
More Harness Programs Do Not Necessarily Produce Useful Specialization
Repeatable score gains from different LLM harness programs are dominated by persistent weaknesses, not useful specialization.
The Chinese University of Hong Kong, New Laboratory of Pattern Recognition (NLPR) · Sept 2026
- Academiccs.RO
Separating exploration and exploitation policies with classifier-free guidance improves diffusion RL fine-tuning
CFG-Policy combines separate exploration and exploitation diffusion policies via classifier-free guidance to improve RL performance while preserving behavioral diversity.
ShanghaiTech University, InstAdapt · Oct 2026
- Top Universitycs.LG
Specialized GFlowNet mixtures find more modes without extra oracle calls
A population of GFlowNets with a learned classifier divides state space to improve mode coverage beyond single-policy training.
Mila – Québec AI Institute, Intel · Oct 2026
- Chinese Techcs.LG
Guided self-exploration lets language models teach themselves better reasoning
SEPO uses labels to guide LLM reasoning exploration, improving self-evolution efficiency and final performance.
Renmin University of China, Peking University · Oct 2026
- Research Labcs.LGcode
Temperature grouping boosts exploration efficiency in LLM reinforcement learning
Temperature-grouped RL converts temperature-driven reward differences into token-level credit, improving RLVR performance and reaching equivalent accuracy up to 36% faster without extra rollouts.
University of Chinese Academy of Sciences, Meituan · Sept 2026
- Big Techcs.LG
Bidirectional start-goal curriculum accelerates sparse-reward reinforcement learning without demonstrations.
Bidirectional Voronoi-biased exploration curricula, growing starts from goals and goals from starts, accelerate sparse-reward goal-conditioned reinforcement learning.
ETH Zürich, NVIDIA · Oct 2026
- Big Techcs.CL
Training a concept generator improves LLM reasoning over repeated sampling.
Training a small concept generator with reinforcement learning nearly doubles a larger frozen model's reasoning pass@k on hard math problems.
Meta FAIR, Université Paris-Saclay · Sept 2026
- Big Techcs.AI
Learning options by focusing on relevant features instead of whole states
Identifying a small relevant subset of features per subgoal yields options that generalize broadly and accelerate exploration.
Amazon, Brown University · Sept 2026
- Top Universitycs.LG
Retry-aware credit assignment boosts LLM reasoning efficiency and accuracy
Retry-aware policy optimization reinforces successful retries without rewarding earlier failures, improving reasoning performance and token efficiency over GRPO.
Columbia University, Yale University · Oct 2026
- Academiccs.CL
Adaptive replay preserves reasoning diversity in RL fine-tuning
Replay of diverse correct base-model solutions during RLVR preserves reasoning diversity without sacrificing performance.
Cornell University · Oct 2026
- Big Techcs.AI
Mined agent skills need domain-specific evidence and format choices
Mined skill performance depends on task domain; workflows beat ontologies on one benchmark, outcome labels help on another, but no configuration wins everywhere.
Massachusetts Institute of Technology, Microsoft Corporation · Oct 2026
- Big Techcs.AI
Simple credit shaping boosts exploration in language model reinforcement learning
ExPPO redistributes reinforcement-learning credit toward surprising responses, improving reasoning coverage and correct-answer diversity while preserving verifier reward direction.
Mila - Quebec AI Institute, Polytechnique Montréal · Oct 2026
- Chinese Techcs.AI
Adaptive feedback helps reinforcement-learning agents target their changing weaknesses
ARISE adapts evaluation criteria and training priorities to an agent's evolving capabilities, improving long-horizon task performance.
ShanghaiTech University, Ant Group · Sept 2026
- Top Universitycs.LG
Rewards Barely Drive Learning in Direct In-Context Reinforcement Learning
Direct in-context reinforcement learning is driven by trajectory imitation and surface form, not by reward semantics, and should be reframed as in-context learning.
Korea Advanced Institute of Science and Technology (KAIST) · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.