The authors built multilingual, multimodal late-interaction models, which represent queries and documents with multiple vectors rather than one vector each.
Shared embeddings let smaller query models search larger models’ document indexes
Krimmel and colleagues report that two distilled retrieval models support mixed-size deployment while achieving strong results across text, image and visual document benchmarks.
Industry
Markus Krimmel · Louis Milliken · Ekaterina Glazkova · Maximilian Schall · Antoine Chaffin · Sedigheh Eslami · +5 more
Perplexity AI
Research Digest··2 min read
6-billion- and 9-billion-parameter retrieval models trained to reproduce representations from the same 18-billion-parameter teacher.
§