Gacha Decoding recasts generation diversity as an instruction-following problem. Instead of relying on the model's token entropy, it asks the model to enumerate semantic decision axes and options, then uses an external random number generator to pick options sequentially. This separates idea enumeration from selection, allowing the model's instruction-following ability to drive diversity.
Across open-ended domains like chat, creative writing, image generation planning, and protein design, Gacha Decoding outperforms prior diversity methods, achieving up to 2.4x Vendi diversity at similar quality and reaching the same number of high-quality modes with 11x fewer samples. It also discovers novel modes that other approaches miss. Notably, diversity improves as the underlying model becomes a better instruction follower, inverting the usual diversity-capability tradeoff.