Generating neural network weights is hard because permuting hidden units changes the parameter vector but not the function, introducing spurious variation. Existing solutions require aligning neurons across networks or using checkpoints from a common training run, which limits applicability. This paper proposes a model with a permutation-equivariant graph meta network that learns directly from unaligned collections of trained networks. The method reproduces the joint statistics of accuracy and similarity of the training networks, generates task-specific networks for different architectures, and works on architectures with hidden widths not seen during training.
Permutation-equivariant flow matching generates neural network weights without alignment
A Graph Meta Network models the distribution of independently trained networks, matching their accuracy, functional similarity, and weight similarity without requiring neuron matching.
Big Tech
Arkadi Piven · Yam Eitan · Guy Bar-Shalom · Fabrizio Frasca · Daniel Cremers · Thomas Dagès · +2 more
Technion – Israel Institute of Technology · Technical University of Munich · Munich Center for Machine Learning · NVIDIA
Research Digest··3 min read
In one line
A permutation-equivariant flow matching model generates neural network weights directly from independently trained networks without alignment.
Before you read this
An idea this paper builds onRead it as
The authors introduce a flow-matching generative model whose velocity field is parameterized by a permutation-equivariant Graph Meta Network.
The authors built a generative model for neural network weights based on .
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.