Latent communication across different LLMs matches or beats text-based transfer at lower compute.

Authors show that a lightweight transformation layer and two-phase training enable effective KV-cache transfer between heterogeneous models, including context-unaware settings where prior methods fail.

PaperIndependentcs.MAarXiv:2606.13594v1 ↗

Siyi Chen · Xiaoyan Zhang · Meng Wu · Jonathan Tremblay · Valts Blukis · Stan Birchfield · +4 more

Research Digest·12 June 2026·2 min read

Read the paper →

Thread:Agent Harness Optimization

What they did

The authors studied KV-cache communication between heterogeneous agents (different sizes of Qwen3 models: 4B, 8B, 14B). They defined two settings: context-aware (receiver sees its own input) and context-unaware (receiver has no input). They analyzed information structure and derived a duality: context-aware benefits from sparse reasoning signals, context-unaware requires dense contextual knowledge. They proposed a lightweight cross-model cache transformation (a small learned linear projection) and a two-phase training: first reconstruct the source model's hidden states, then train on generation tasks. They evaluated on 6 in-domain and out-of-domain benchmarks across all 6 directions of the three models.

Key findings

In context-aware settings, the method outperforms prior heterogeneous baselines (e.g., Latent Causal Inference, LCI) and matches or exceeds text-based communication at 2-3x lower compute cost.
In context-unaware settings, prior methods collapsed (e.g., LCI dropped to near-random), while the proposed method maintained effectiveness, achieving 70-80% of text-based accuracy.
The two-phase training (reconstruction then generation) was crucial: reconstruction alone was insufficient, generation alone led to poor alignment.
The transformation layer is lightweight (few million parameters) and generalizes to out-of-domain tasks.

Why it matters

This work challenges the assumption that heterogeneous LLMs cannot directly share latent representations. It opens the possibility of more efficient multi-agent systems where agents of different sizes and architectures can communicate via compressed latent channels, reducing the need for explicit text decoding. This could enable faster, lower-latency collaboration in distributed AI systems.

Caveats

The experiments are limited to models from the same family (Qwen3) and sizes up to 14B. Cross-family transfer (e.g., Qwen to Llama) was not tested. The method still requires fine-tuning the transformation layer per pair of models, which may not scale to large heterogeneous fleets. The compute savings are relative to text-based communication, but the training cost for alignment is non-trivial.

ZOTPAPER