Open-Ended RL Reward Design
Methods for constructing rewards for reinforcement learning on tasks with no single correct answer, using relative comparisons or rubrics.
44 papers · 1 months
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
3 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Oct 2026 · MIT
Joint scaling with data and budget governs reward optimization performance
Provides an information-theoretic scaling law for preference-based reward optimization, informing reward design under data and budget constraints.
4 further papers
Oct 2026 · Accenture Strategy & Consulting, Vizuara AI Labs
Pass@k misses output diversity changes caused by model post-training
Shows that post-training methods change output diversity measured by token entropy, answer entropy, and unique-answer counts, which is relevant for reward design on tasks with no single correct answer.
Oct 2026 · Korea Advanced Institute of Science and Technology (KAIST)
Rewards Barely Drive Learning in Direct In-Context Reinforcement Learning
Empirically demonstrates that rewards exert minimal influence on learning in direct in-context RL, questioning reward design assumptions.
1 further paper
Oct 2026 · Massachusetts Institute of Technology, Cornell University
Reward models for vision-language AI can be trained without human preference labels
Builds a vision-language reward model by ranking web captions above perturbed versions on a frozen backbone, with no human preference labels.
1 further paper
Oct 2026 · Microsoft Research, Princeton University
Optimizing AI persuasion rewards fabrication unless alignment is explicitly enforced
this paper demonstrates that RL optimization for persuasion rewards can amplify misaligned fabrication, highlighting the need for explicit alignment constraints in reward design.
3 further papers
Oct 2026 · Massachusetts Institute of Technology, Microsoft Research
Adversarial rubric repair improves self-training on open-ended expert tasks
Presents adversarial rubric repair for self-training on open-ended tasks, addressing reward hacking in rubric-based RL.
4 further papers
Oct 2026 · AWS AI Labs, University of Massachusetts
Minimal corrections teach hard tasks while preserving broader model skills
Uses minimal corrections to provide learning signal for tasks with no successful attempts, addressing reward signal scarcity.
2 further papers
Oct 2026 · Google DeepMind
Rewarding moral arguments makes a language model more consistent, not sounder
Trains a model via reward from its own moral comparisons and finds increased consistency but not soundness
7 further papers
Oct 2026 · Edison Scientific Inc.
AI reasoning models learn from reference examples alone, no reward needed
Proposes Reinforced Adversarial Learning, using a judge trained on reference outputs to provide reward signal.
released code
3 further papers
Nov 2026 · University of Illinois Urbana-Champaign, National University of Singapore
AI agents picking from a menu succeed more often with less training
Trains an action-probability head over action menus rather than generating text, reformulating the RL reward signal to focus on correct choice rather than token-level generation.
Nov 2026 · Mohamed bin Zayed University of Artificial Intelligence
Language models’ responses to reward changes can be forecast before training
Forecasts how language model responses change under reward edits, aiding reward design.
Nov 2026 · Stanford University, HKUST (Guangzhou)
AI protein design improves by separating suggested changes from evaluation
Separates suggestion signals from evaluation in protein design search, informing reward design for open-ended tasks.
Nov 2026 · University of Cambridge, Qwen Business Unit, Alibaba
Budgeting rewards per prompt helps language models train past reasoning plateaus
Shows per-prompt reward budgeting helps RL escape reasoning training plateaus where entropy bonuses do not.
3 further papers
Nov 2026 · University of California, San Diego, Google Inc.
User-agnostic reward component, not the personalized one, narrows AI recommender picks
Finds that collaborative-filtering rewards make LLM recommenders concentrate on fewer items, highlighting a reward design pitfall.
1 further paper
Nov 2026 · University of Washington, University of California, Berkeley
Multi-agent teams learn to organize themselves, beating solo agents by 25×
Introduces a multi-agent RL method with emergent self-organization, demonstrating a novel reward design for coordination tasks.
15 of 44 papers shown