cross-modal transfer
- Papers
- 4
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Nov 2026
4 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Audio training teaches one model to edit scenes across modalities
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
Oct 2026 - 2Voice chatbots can lose conversational abilities they demonstrate with text
Transcript performance is an unreliable proxy for multilingual spoken conversational ability, especially because it masks failures in tracking spoken history.
Nov 2026 - 3Image models can outperform specialized models at reading brain recordings
A frozen vision model that has never seen EEG outperforms specialized EEG models when signals are rendered as images.
Nov 2026
How fast it is moving
Often studied with
self-supervised learningA paradigm where models learn representations from unlabeled data by predicting hidden or transformed parts of the input.2 togethercompute-optimal scalingThe practice of scaling model size and data volume to achieve optimal performance for a given compute budget.1 togetherdata repetitionReusing the same training samples across epochs, affecting overfitting, model degradation, and scaling-law predictions in pretraining.1 togetherThe papers
Most central to this idea first, not most recent.
- Academiccs.AI
Image models can outperform specialized models at reading brain recordings
A frozen vision model that has never seen EEG outperforms specialized EEG models when signals are rendered as images.
Rutgers University–New Brunswick · Nov 2026
- Big Techcs.SD
Audio training teaches one model to edit scenes across modalities
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
Adobe Research, Carnegie Mellon University · Oct 2026
- Chinese Techcs.CL
Voice chatbots can lose conversational abilities they demonstrate with text
Transcript performance is an unreliable proxy for multilingual spoken conversational ability, especially because it masks failures in tracking spoken history.
Nankai University, Alibaba Token Hub, Alibaba Group · Nov 2026
- Top Universitycs.CL
Multimodal training needs different scaling recipes for text, vision and audio
Scaling laws for multimodal pretraining reveal modality-specific compute optima, asymmetric interference, and saturation of repeated non-text data.
MILA - Quebec AI Institute, McGill University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.