The authors separated communication into context encoding and question-dependent reading.
Compressed latent packets let different language models reuse context efficiently
LatComm encodes a document once into a fixed-size representation that frozen receiver models can query differently for each question.
Big Tech
Huawei Lin · Jinting Liu · Tan Yu · Weijie Zhao
Rochester Institute of Technology · NVIDIA
Research Digest··3 min read
Lin and colleagues introduce LatComm, an interface designed to prevent language models from repeatedly processing the same long context.
§