The authors measured how changes to early patch features affect the encoder output.
A frozen vision encoder yields a robust perceptual image distance
JLD weights spatially detailed early features by their influence on later representations, without fitting the metric to human ratings.
Big Tech
Shreshth Saini · Balu Adsumilli · Alan C. Bovik
The University of Texas at Austin · Google · University of Colorado Boulder
Research Digest··2 min read
Saini, Adsumilli and Bovik derive an image-distance metric from the Jacobian, or local sensitivity, of a frozen DINOv2-S vision encoder.
Why this paper
From Google and 2 others
In one line
A frozen vision encoder's Jacobian weights early patch features to create a perceptual distance that matches human judgments without human labels.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§