The authors study a selector that receives n training examples, optionally v validation examples, and must choose a smaller training subset.
Data selection faces statistical limits, but can target full-training fidelity
The authors establish a worst-case barrier to identifying optimal training subsets and propose a gradient-based bound for preserving full-data training behavior.
Big Tech
Ahmed Mehdi Inane · Mahdi Haghifam · Gintare Karolina Dziugaite · Ioannis Mitliagkas
Université de Montréal · Mila · Toyota Technological Institute at Chicago · Google DeepMind
Research Digest··3 min read
Inane and colleagues distinguish finding the best training subset from finding one that behaves like training on all available data.
§