Representation-Based Distillation
Investigates methods where larger language models share internal representations (rather than output logits or natural language) to improve the performance of smaller models.
2 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.