The authors curated a spatial audio-visual dataset with true stereo acoustics and metric camera poses from web videos, game recordings, and open corpora.
Interactive world model adds synchronized spatial audio to visual simulation
HelixWorld generates camera-grounded stereo sound and video in real time, matching silent models in visual quality while improving acoustic immersion.
Academic
Lei Ke · Jiahao Pan · Zeyue Tian · Jiaming Wang · Haoyuan Huang · Kam Man Wu · +10 more
The Hong Kong University of Science and Technology · Noiz AI
Research Digest··2 min read
The authors introduce HelixWorld, the first real-time interactive audio-visual world model that co-generates visual scenes and camera-grounded spatial stereo audio under user control.
Why this paper
From The Hong Kong University of Science and Technology and Noiz AI · Released code
In one line
HelixWorld generates controllable visual scenes and camera-aligned stereo audio at 24 FPS on one GPU while maintaining long-horizon synchronization.
What it released
Code
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (hardware NVIDIA H800)
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§