The authors evaluated the finite-sample sensitivity of the default kernel metrics KID (cubic polynomial kernel) and KAD (Gaussian kernel with median bandwidth) on real embedding distributions: CLaMP 2 for symbolic music, VGGish and PANNs for audio, and Inception features of CIFAR-10 and STL-10 for images.
Default kernel metrics for generative models miss rotation changes at common sample sizes
Radzikowski and Modrzejewski show that the standard KID and KAD kernel configurations detect mean shifts reliably but rarely detect covariance-preserving rotations on real embeddings at 780 samples per side.
Academic
Tomasz Radzikowski · Mateusz Modrzejewski
Warsaw University of Technology
Research Digest··3 min read
The authors tested the default kernels of KID and KAD as permutation two-sample tests on real embeddings of symbolic music, audio and images, applying controlled changes (mean shift, isotropic contraction, rotation) matched in 2-Wasserstein distance.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.