cross-modal transfer
- Papers
- 3
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Nov 2026
3 papers in the last two months, against 0 in the two before.
Start here
Two papers to read first, in the order they came out- 1Audio training teaches one model to edit scenes across modalities
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
Oct 2026 - 2Voice chatbots can lose conversational abilities they demonstrate with text
Transcript performance is an unreliable proxy for multilingual spoken conversational ability, especially because it masks failures in tracking spoken history.
Nov 2026
How fast it is moving
Often studied with
benchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.1 togethercompute-optimal scalingThe practice of scaling model size and data volume to achieve optimal performance for a given compute budget.1 togetherdata repetitionReusing the same training samples across epochs, affecting overfitting, model degradation, and scaling-law predictions in pretraining.1 togetherThe papers
Most central to this idea first, not most recent.
- Big Techcs.SD
Audio training teaches one model to edit scenes across modalities
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
Adobe Research, Carnegie Mellon University · Oct 2026
- Chinese Techcs.CL
Voice chatbots can lose conversational abilities they demonstrate with text
Transcript performance is an unreliable proxy for multilingual spoken conversational ability, especially because it masks failures in tracking spoken history.
Nankai University, Alibaba Token Hub, Alibaba Group · Nov 2026
- Top Universitycs.CL
Multimodal training needs different scaling recipes for text, vision and audio
Scaling laws for multimodal pretraining reveal modality-specific compute optima, asymmetric interference, and saturation of repeated non-text data.
MILA - Quebec AI Institute, McGill University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.