DrivingBench is a new where general-purpose drive a real Toyota Corolla around a parking lot cone course. The models see camera frames and directly command steering and velocity through three MCP tools, while the car continues moving during inference, making latency part of the task.
Four frontier models were tested: GPT-6 Astra, Claude Fable 5.1, GPT-5.6 Sol, and Grok 4.6. Only GPT-6 Astra finished the 127-meter course, completing it on its second attempt in 4 minutes and 44 seconds. No other model passed 50% of the course; Claude Fable 5.1 reached 45% on its third attempt, while the others failed the first corner. Two models improved materially across attempts with retained context.
This is interesting because it tests real-world physical capabilities out-of-the-box in frontier models, rather than digital benchmarks. It shows that general-purpose VLMs are beginning to act in the physical world, but with significant failures in perception, planning, and latency management. The benchmark and its traces are released for reproducibility.