Matching the same object part across different photos is challenging because appearances vary widely. Current methods find the correct match among the top candidates, but often pick the single closest match incorrectly. The problem is not the features, but the selection rule. The authors propose a test-time re-ranking module. It first adaptively selects a set of candidate matches, then scores them using cycle consistency: a good match from source to target should strongly match back to the source. No retraining is needed. This approach consistently improves performance on several benchmarks. The key insight is that improving the final decoding step, rather than just the features, offers a simple and effective path to better semantic matching.
Correct image correspondences often in top-k but missed by top-1
A training-free reranking module using cycle consistency improves matching accuracy by up to 4.9 percentage points on standard benchmarks.
Yale University · Amazon.com, Inc. · University of Rochester · Georgia Institute of Technology · Australian National University
In one line
Correct semantic correspondences are often in the top candidates but missed by greedy top-1 selection, which cycle-consistent re-ranking fixes.
Before you read this
An idea this paper builds onThe authors first quantified a "decoding bottleneck" in semantic correspondence: on SPair-71k, with 30 candidates, zero-shot features from large pretrained models exceed the top-1 accuracy of a fully supervised baseline, yet top-1 selection misses these correct matches.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.