← All threads

Open-Ended RL Reward Design

Methods for constructing rewards for reinforcement learning on tasks with no single correct answer, using relative comparisons or rubrics.

44 papers · 1 months

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

Results across this thread

3 reported results from the papers in this thread.

PaperBenchmarkMetricResultConditionsCode

The table is for subscribers. Subscribe to see every reported number side by side.

How this thread developed

  1. Oct 2026 · MIT

    Joint scaling with data and budget governs reward optimization performance

    Provides an information-theoretic scaling law for preference-based reward optimization, informing reward design under data and budget constraints.

  2. 4 further papers

    Oct 2026 · Accenture Strategy & Consulting, Vizuara AI Labs

    Pass@k misses output diversity changes caused by model post-training

    Shows that post-training methods change output diversity measured by token entropy, answer entropy, and unique-answer counts, which is relevant for reward design on tasks with no single correct answer.

  3. Oct 2026 · Korea Advanced Institute of Science and Technology (KAIST)

    Rewards Barely Drive Learning in Direct In-Context Reinforcement Learning

    Empirically demonstrates that rewards exert minimal influence on learning in direct in-context RL, questioning reward design assumptions.

  4. 1 further paper

    Oct 2026 · Massachusetts Institute of Technology, Cornell University

    Reward models for vision-language AI can be trained without human preference labels

    Builds a vision-language reward model by ranking web captions above perturbed versions on a frozen backbone, with no human preference labels.

  5. 1 further paper

    Oct 2026 · Microsoft Research, Princeton University

    Optimizing AI persuasion rewards fabrication unless alignment is explicitly enforced

    this paper demonstrates that RL optimization for persuasion rewards can amplify misaligned fabrication, highlighting the need for explicit alignment constraints in reward design.

  6. 3 further papers

    Oct 2026 · Massachusetts Institute of Technology, Microsoft Research

    Adversarial rubric repair improves self-training on open-ended expert tasks

    Presents adversarial rubric repair for self-training on open-ended tasks, addressing reward hacking in rubric-based RL.

  7. 4 further papers

    Oct 2026 · AWS AI Labs, University of Massachusetts

    Minimal corrections teach hard tasks while preserving broader model skills

    Uses minimal corrections to provide learning signal for tasks with no successful attempts, addressing reward signal scarcity.

  8. 2 further papers

    Oct 2026 · Google DeepMind

    Rewarding moral arguments makes a language model more consistent, not sounder

    Trains a model via reward from its own moral comparisons and finds increased consistency but not soundness

  9. 7 further papers

    Oct 2026 · Edison Scientific Inc.

    AI reasoning models learn from reference examples alone, no reward needed

    Proposes Reinforced Adversarial Learning, using a judge trained on reference outputs to provide reward signal.

    released code

  10. 3 further papers

    Nov 2026 · University of Illinois Urbana-Champaign, National University of Singapore

    AI agents picking from a menu succeed more often with less training

    Trains an action-probability head over action menus rather than generating text, reformulating the RL reward signal to focus on correct choice rather than token-level generation.

  11. Nov 2026 · Mohamed bin Zayed University of Artificial Intelligence

    Language models’ responses to reward changes can be forecast before training

    Forecasts how language model responses change under reward edits, aiding reward design.

  12. Nov 2026 · Stanford University, HKUST (Guangzhou)

    AI protein design improves by separating suggested changes from evaluation

    Separates suggestion signals from evaluation in protein design search, informing reward design for open-ended tasks.

  13. Nov 2026 · University of Cambridge, Qwen Business Unit, Alibaba

    Budgeting rewards per prompt helps language models train past reasoning plateaus

    Shows per-prompt reward budgeting helps RL escape reasoning training plateaus where entropy bonuses do not.

  14. 3 further papers

    Nov 2026 · University of California, San Diego, Google Inc.

    User-agnostic reward component, not the personalized one, narrows AI recommender picks

    Finds that collaborative-filtering rewards make LLM recommenders concentrate on fewer items, highlighting a reward design pitfall.

  15. 1 further paper

    Nov 2026 · University of Washington, University of California, Berkeley

    Multi-agent teams learn to organize themselves, beating solo agents by 25×

    Introduces a multi-agent RL method with emergent self-organization, demonstrating a novel reward design for coordination tasks.

15 of 44 papers shown