← All concepts

group relative policy optimization

To train a language model without a separate value network, you can generate several answers for the same prompt and reward each one according to how it ranks against the others. That is the core idea: relative, group-level feedback. It is cheaper than typical reinforcement learning setups and works well when you have a way to score answers, such as checking mathematical or code correctness.

The papers you shared all start from this setup and try to fix its weaknesses. Common fixes include giving credit per token instead of per whole sequence, discarding groups where every answer is equally good or bad, adding extra teacher guidance to failed attempts, or tailoring updates to similar users. A recurrent finding is that stable and bounded reward signals help more than complex or noisy ones, and that these modifications often speed up training while preserving or improving final performance.

Papers
182
Released code
11
First seen
June 2026
Latest
Nov 2026

175 papers in the last two months, against 6 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Merging specialist AI models saves 42% of training compute compared to joint training

    On-policy distillation beats GRPO in compute efficiency only when using the student's own GRPO checkpoint as teacher.

    Nov 2026
  2. 2AI models can read pictured problems yet fail to solve them

    Multimodal LLMs fail to reason on images they can read because a reasoning-access gap blocks visual input from using text reasoning machinery.

    Nov 2026
  3. 3Dense rewards for language models lose to simpler reward when compute is matched

    Matched-compute comparisons show dense process rewards fail to beat baseline GRPO, while prompt selection yields a real gain.

    Nov 2026

Who is working on it

Zhejiang University 12Peking University 9Tsinghua University 8University of Science and Technology of China 8University of Illinois Urbana-Champaign 8Alibaba Group 8

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.