← All concepts

reward model

Imagine an automated judge that scores how good an AI's output is, based on what humans prefer. A "reward model" is that judge. It learns from examples of human ratings to predict a score for any new output. This score can then guide the AI to produce better results, whether for text, video, or search.

The papers all work on making these judges more accurate or useful. They show that replacing a simple score with a richer prediction, such as a distribution, a temporal alignment, or a detailed evaluation, improves agreement with humans and helps AI perform better. They also reveal hidden risks: reward models can learn harmful preferences if poisoned, and their effectiveness depends on how uncertainty is handled.

Papers
6
Released code
0
First seen
Oct 2026
Latest
Oct 2026

6 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Interactive software judges improve when they test applications directly

    Interaction-as-a-Judge using computer-use agents improves evaluation and generation of interactive applications over code-only methods.

    Oct 2026
  2. 2Human-rated clips improve rewards for synchronizing generated sound and video

    SyncReward is a human-aligned reward model that improves audio-visual synchronization in generated content.

    Oct 2026
  3. 3Narrow harmful training can broadly invert reward model preferences

    Narrow harmful fine-tuning makes reward models prefer harmful answers across unseen domains while preserving most benchmark performance through compression followed by harm-specific rotation.

    Oct 2026

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.