Extreme Model Pruning
Methods for removing most model weights, often with lightweight recovery training, while preserving prediction quality.
4 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
8 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Nov 2026 · University of Bath, Aalto University
Language models can lose 90% of weights with cheaper pruning
Demonstrates gradual pruning with lightweight adjustment training can remove 90% of weights more cheaply than prior pruning.
1 further paper
Nov 2026 · Amazon Web Services
Combining model compression techniques can collapse small language models
Studies the interaction of pruning with quantization and fine-tuning, revealing degradation when techniques are combined on small models.
Nov 2026 · Amazon, Amazon Web Services
Language models can merge specialists without matching their training capacity
Finds that merging specialists does not require matching training adapter capacity, tying model merging to pruning concepts.
3 of 4 papers shown