DSL moves collaboration from parameter space to function space.
Random Outputs Can Replace Model Updates in Collaborative Learning
The authors distill task-unrelated model outputs to combine specialized language models and train federated classifiers with substantially less uplink communication.
Top University
Dario Fenoglio · Gabriele Dominici · Martin Gjoreski · Marc Langheinrich
Università della Svizzera italiana (USI) · MIT Media Lab · Massachusetts Institute of Technology
Research Digest··2 min read
Fenoglio and colleagues introduce Distributed Subliminal Learning (DSL), in which participants probe locally adapted models with random, task-unrelated inputs and send the resulting outputs instead of parameters.
Why this paper
From MIT Media Lab and 2 others
In one line
Knowledge can be shared between models by transmitting only outputs on random, task-unrelated inputs.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§