The authors reframe internal-state sharing in multi-agent language-model systems as a pool mechanism: an agent publishes its internal state once, and any peer reads it through a single mechanism fitted to no particular sender.
For internal-state sharing between language models, augment the cache and replace the activations
A controlled study of two carriers and two fusion mechanisms shows that which combination works depends on where the representation enters the receiver, challenging the prevailing view that carrier and fusion can be chosen independently.
Big Tech
Adam Deng · Shicheng Liu · Yada Zhu
MIT · IBM Research
Research Digest··3 min read
Deng, Liu, and Zhu systematically compared two carriers of internal state (KV cache and residual-stream activations) and two fusion mechanisms (augmentation and replacement) in a controlled latent-fact transfer setting.
Why this paper
From IBM Research and MIT
In one line
For internal state sharing between frozen agents, augment the KV cache and replace the activation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§