The authors introduce CreativePreferences, spanning mathematics, coding, law, academia, journalism, creative writing and humor.
Human preferences contain signals that rubrics and verifiers miss
Across seven creative and technical domains, dense preference models captured human judgments that executable checks and verbal criteria could not.
Top University
Alexander Spangher · Sheldon Huang · Andreas Haupt · Noah D. Goodman · Diyi Yang · Daniel E. Ho · +1 more
Stanford University · University of Toronto
Research Digest··3 min read
Thread:LLM Judge Decomposition
Spangher et al.
Why this paper
From Stanford University and University of Toronto · Part of LLM Judge Decomposition, now 3 papers
In one line
Preference judgments contain tacit components not captured by articulated rules or verifiable checks, across all domains including math and code.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§