← All threads

Training Data Subset Selection

Investigates methods and theoretical limits of selecting subsets of training data to match full-dataset training fidelity.

7 papers

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

Results across this thread

16 reported results from the papers in this thread.

PaperBenchmarkMetricResultConditionsCode

The table is for subscribers. Subscribe to see every reported number side by side.

How this thread developed

  1. Oct 2026 · Université de Montréal, Mila

    AI cannot pick optimal training data without much larger evaluation sets

    Identifies statistical limits of data selection and proposes targeting full-training fidelity as a more feasible objective.

  2. Oct 2026 · University of Sydney

    Top-k data scores under-cover latent modes; class balancing or coverage fixes selection

    Proves that fixed top-k selection incurs a closed-form generalization gap and proposes coverage fixes.

  3. 1 further paper

    Oct 2026 · JD.com, University of Oxford

    Data curation advances LLM training only if a train-target gap exists

    Formalizes advancement vs acceleration effects of data curation and proves theoretical condition for true advancement.

  4. 2 further papers

    Oct 2026 · Northeastern University, MIT

    Training data attribution for AI language models degrades with redundancy and distractors

    introduces CATT benchmark to evaluate data attribution methods under redundancy and distractors.

4 of 7 papers shown