Training Data Subset Selection
Investigates methods and theoretical limits of selecting subsets of training data to match full-dataset training fidelity.
7 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
16 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Oct 2026 · Université de Montréal, Mila
AI cannot pick optimal training data without much larger evaluation sets
Identifies statistical limits of data selection and proposes targeting full-training fidelity as a more feasible objective.
Oct 2026 · University of Sydney
Top-k data scores under-cover latent modes; class balancing or coverage fixes selection
Proves that fixed top-k selection incurs a closed-form generalization gap and proposes coverage fixes.
1 further paper
Oct 2026 · JD.com, University of Oxford
Data curation advances LLM training only if a train-target gap exists
Formalizes advancement vs acceleration effects of data curation and proves theoretical condition for true advancement.
2 further papers
Oct 2026 · Northeastern University, MIT
Training data attribution for AI language models degrades with redundancy and distractors
introduces CATT benchmark to evaluate data attribution methods under redundancy and distractors.
4 of 7 papers shown