The authors reformulated cross-architecture weight transformation as a two-input operation.
Graph metanetworks accelerate compression across different neural network architectures
CrossGMN predicts target-network weights from a trained source and a target initialization while respecting each network’s permutation symmetries.
Big Tech
Adir Dayan · Yam Eitan · Haggai Maron
Technion – Israel Institute of Technology · NVIDIA Research
Research Digest··2 min read
Dayan, Eitan and Maron introduce CrossGMN, a metanetwork that jointly represents a trained source model and a differently shaped target model as a connected graph.
Why this paper
From NVIDIA Research and Technion – Israel Institute of Technology
In one line
CrossGMN maps weights between different neural architectures while respecting permutation symmetries, accelerating knowledge distillation by up to 8.89x.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§