Blind and low-vision viewers often miss visual details in online videos. Audio descriptions narrate key visuals to fill that gap, but they must fit into natural pauses without overlapping spoken audio. Until now, there was no good way to test how well automated descriptions do this on everyday user videos.
The researchers built a dataset of 433 videos with professionally crafted audio descriptions, co-written by blind and sighted experts. They then created an evaluation framework that converts generated descriptions into spoken audio and measures how well they slot into the video's silences. They tested several large multimodal models on this .
No model matched human pacing, but the best models reduced overlapping speech to under a third of a second per video. Blind participants found the automated descriptions highly helpful for understanding, even though they still preferred human-written ones. The benchmark and evaluation tools are freely released.