Generating real-time audio from live video demands both speed and accurate synchronization. Existing streaming methods either use video encoders that discard context, hurting alignment, or slow ones that cause delay. They also generate each audio chunk independently, reducing audio quality.
LiveFoley overcomes this with a trailing sliding window that lets powerful video encoders run with only 0.08 seconds of look-ahead. Its denoising module uses previously generated chunks for better sound, and can include a future chunk for even higher quality.
This matches offline models in synchronization while keeping total delay to 156 milliseconds. That keeps the audio-visual gap below what humans perceive, making it suitable for interactive worlds like games and VR.