Extreme Model Pruning
Methods for removing most model weights, often with lightweight recovery training, while preserving prediction quality.
5 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
8 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Nov 2026 · University of Bath, Aalto University
Language models can lose 90% of weights with cheaper pruning
Demonstrates gradual pruning with lightweight adjustment training can remove 90% of weights more cheaply than prior pruning.
2 further papers
Nov 2026 · University of Southern California, University of Oxford
Random weight perturbations don't reliably find task experts in pretrained models
Tests and refutes the reliability of random-weight-perturbation for identifying task experts, informing pruning approaches.
Nov 2026 · Amazon, Amazon Web Services
Language models can merge specialists without matching their training capacity
Finds that merging specialists does not require matching training adapter capacity, tying model merging to pruning concepts.
3 of 5 papers shown