AudioFM uses two complementary views (time- and frequency-masked) for global contrastive learning, and a local objective where a student predicts representations of masked patches and identifies their corresponding full-view teacher targets among candidates from other recordings.
Joint global-local discrimination yields general-purpose audio representations
AudioFM combines recording-level contrast with masked patch prediction, outperforming prior models on 12 tasks spanning environmental, speech, music, and medical audio.
Big Tech
Yamin Li · Ruidong Zhang · Anran Wang · Eduardo Fonseca · Dan Ellis · Zechen Li · +16 more
Google Research · Google DeepMind · Vanderbilt University · University of Washington
Research Digest··2 min read
The authors introduce AudioFM, a self-supervised framework that learns audio representations by combining recording-level contrastive learning with a patch-wise masked latent discrimination objective.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.