reward model
Imagine an automated judge that scores how good an AI's output is, based on what humans prefer. A "reward model" is that judge. It learns from examples of human ratings to predict a score for any new output. This score can then guide the AI to produce better results, whether for text, video, or search.
The papers all work on making these judges more accurate or useful. They show that replacing a simple score with a richer prediction, such as a distribution, a temporal alignment, or a detailed evaluation, improves agreement with humans and helps AI perform better. They also reveal hidden risks: reward models can learn harmful preferences if poisoned, and their effectiveness depends on how uncertainty is handled.
- Papers
- 6
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Oct 2026
6 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Interactive software judges improve when they test applications directly
Interaction-as-a-Judge using computer-use agents improves evaluation and generation of interactive applications over code-only methods.
Oct 2026 - 2Human-rated clips improve rewards for synchronizing generated sound and video
SyncReward is a human-aligned reward model that improves audio-visual synchronization in generated content.
Oct 2026 - 3Narrow harmful training can broadly invert reward model preferences
Narrow harmful fine-tuning makes reward models prefer harmful answers across unseen domains while preserving most benchmark performance through compression followed by harm-specific rotation.
Oct 2026
How fast it is moving
Often studied with
agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real2 togetherreinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.2 togetherbenchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.1 togetherThe papers
Most central to this idea first, not most recent.
- Academiccs.AI
Narrow harmful training can broadly invert reward model preferences
Narrow harmful fine-tuning makes reward models prefer harmful answers across unseen domains while preserving most benchmark performance through compression followed by harm-specific rotation.
Northeastern University · Oct 2026
- Independentcs.LG
Interactive software judges improve when they test applications directly
Interaction-as-a-Judge using computer-use agents improves evaluation and generation of interactive applications over code-only methods.
Oct 2026
- Chinese Techcs.CV
Human-rated clips improve rewards for synchronizing generated sound and video
SyncReward is a human-aligned reward model that improves audio-visual synchronization in generated content.
Renmin University of China, WeChat, Tencent Inc. · Oct 2026
- Chinese Techcs.AI
Grounding rewards must check source independence and conflict handling in search agents
A hierarchical reward for source independence and conflict handling improves search-agent evidence grounding beyond outcome and citation checks.
Peking University, Tencent · Oct 2026
- Top Universitycs.LG
Diffusion reward models capture multimodal preference that scalar scores miss
DRM models reward as a multimodal conditional density, matching or surpassing baseline reward models.
Tsinghua University, The Chinese University of Hong Kong · Oct 2026
- Top Universitycs.LG
Confidence filtering gives beam search stronger test-time scaling guarantees
Confidence-filtered beam search reduces test-time sample complexity from quadratic to nearly linear and provably outperforms sequence-level inference.
The Ohio State University, University of Pennsylvania · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.