The authors developed AnchorDOT, a three-stage framework for aligning two independently trained, frozen encoders.
Optimal transport aligns pretrained modalities with only hundreds of paired anchors
AnchorDOT uses 100 to 500 paired examples plus independently sampled unpaired data to connect frozen image, text and molecular representation spaces.
Research Lab
Julien Kordes · Reuben Dorent
Sorbonne Université · Institut du Cerveau - Paris Brain Institute - ICM · CNRS · Inria · Inserm
Research Digest··2 min read
Kordes and Dorent address cross-modal alignment when paired data are scarce but pretrained unimodal encoders and unpaired samples are plentiful.
Why this paper
From Inria and 6 others
In one line
Aligning two frozen unimodal encoders requires only 100 to 500 paired anchors when anchor-relative coordinates, entropic optimal transport, and distillation are combined.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§