Large language models (LLMs) are AI programs that generate text. When several LLMs work together as a team, each keeps a memory of its conversation called a key-value cache. These caches use a lot of memory, especially when many models run at once.
CoMixKV gives different precision to different parts of the cache. It identifies which parts are more important and gives them higher precision, while staying within a total memory limit. This is done automatically as the models run, without extra test data.
In tests, it cut peak memory by about a third and kept accuracy high. This lets multi-agent LLM systems run more efficiently on existing hardware.