The authors formulate continual multi-teacher distillation, in which foundation models arrive sequentially and earlier teachers are no longer available.
One vision backbone can continually absorb new foundation model capabilities
GRAFT adds heterogeneous visual skills sequentially while using the previous student, rather than all earlier teachers, to preserve existing representations.
Big Tech
Zhenghao Zhao · Chi Zhang · Qingshuang Chen · Yelin Kim
University of Illinois Chicago · Amazon
Research Digest··3 min read
Zhao et al.
Why this paper
From Amazon and University of Illinois Chicago
In one line
GRAFT continually extends a vision backbone by distilling from new teachers while using the previous model to retain old capabilities.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§