The authors exploited the fact that audio sources can be added as waveforms to create physically plausible mixtures.
Audio training teaches one model to edit scenes across modalities
CrossEdit transfers compositional instruction-following learned from synthetic audio mixtures to previously unseen audiovisual movie-scene edits.
Big Tech
William Chen · Prem Seetharaman · Ke Chen · Oriol Nieto · Kevin Duarte · Siddharth Srinivasan Iyer · +7 more
Adobe Research · Carnegie Mellon University
Research Digest··3 min read
Chen and colleagues developed CrossEdit, a unified model for editing images, audio and video from free-form instructions.
Why this paper
From Adobe Research and Carnegie Mellon University
In one line
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§