When fine-tuning a (LLM), you often can't use all your training examples, so you pick a subset. Older methods favored the hardest examples, assuming they teach the most. That assumption came from smaller classification models, which predict from a fixed label set.\n\nTesting five difficulty scores for LLM fine-tuning, the authors found they often did worse than random selection. They built a simple method that groups examples by similarity and picks examples from each group, making the subset mirror the whole dataset.\n\nThis coverage-based selection matched or beat random selection, and on a highly diverse multi-domain dataset it beat all methods. For LLMs, representativeness matters more than difficulty: choose data that covers the range of your input space.
Data coverage beats difficulty for choosing language model fine-tuning data
Randomly selecting training examples often outperforms methods that pick the hardest ones, because random selection better represents the full input distribution.
University of Chicago · The University of Texas at Austin · Allen Institute for Artificial Intelligence
In one line
For LLM fine-tuning, selecting training data for coverage of the input distribution beats selecting for difficulty, matching or beating random selection.
Before you read this
An idea this paper builds onThe authors tested five established difficulty scores (e.g., forgetting events, self-supervised loss, EL2N) for selecting fine-tuning data for two language models (RoBERTa-large and LLaMA-2 7B) on three tasks where zero-shot performance is weak: relation extraction, natural language inference, and question answering.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.