Distributed Subliminal Learning (DSL) is a method where participants adapt a common model locally, then instead of sending model updates, they send only the model's outputs on random, task-unrelated inputs (carriers). A central coordinator pools these outputs and trains a shared model via distillation. This approach avoids the communication cost and weight-space interference typical of parameter exchange.
In large language model composition, DSL achieved 94.56% preference retention compared to 87.76% for LoRA averaging, and a 22-point gain on GSM8K versus only 0.6 for the baseline, while reducing upload size by 30-49x. In federated learning for image classification, DSL nearly matched FedAvg on MNIST with 8.9x less uplink and provided lower-communication options on CIFAR-10 and Tiny ImageNet. The work shows that useful knowledge can be transmitted through model behavior on irrelevant inputs.