Web agents are AI programs that complete tasks in a browser, like scheduling meetings. Current tests cover few website types, exclude audio and video, and cannot be reproduced exactly.
The researchers built WebArena-Pro, a with hundreds of tasks across twenty self-hosted apps in six domains. It adds audio and video actions, and runs each task in isolation for reproducibility.
The best agent succeeded on only 62.3 percent of tasks within a fifty-step budget, showing agents struggle with diverse interfaces and media.
