The authors surveyed 82 papers on LLM creativity evaluation and organized their designs into a taxonomy.
LLM creativity is best measured by performance on rare, long-tail outputs
Across 15 benchmarks and 26 models, the authors show creativity evaluations probe low-probability regions and tail-only rankings better match human creative-writing judgments.
Big Tech
Yichen Wang · Tenghao Huang · Vishakh Padmakumar · Mina Lee
University of Chicago · Microsoft · Stanford University
Research Digest··3 min read
The authors propose that what creativity benchmarks share is the evaluation of task performance in the long tail of a reference input-output distribution, where inputs or outputs are improbable.
Why this paper
From Microsoft and 2 others
In one line
LLM creativity benchmarks share a common target: performance on rare input-output pairs in the long tail of a reference text distribution.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§