The authors curated 112 expert-authored tasks built on XR Blocks, an open-source WebXR framework: 24 feature implementations, 21 bug repairs, and 67 end-to-end application generation tasks.
First benchmark evaluates coding agents on XR development tasks
GenXR-Bench contains 112 expert-authored tasks and finds best model achieves only 50.9% Pass@1, with frequent errors in coordinate transformations and sensor handling.
Big Tech
William Huang · David Li · Xiang 'Anthony' Chen · Xun Qian · Nels Numan · Zhongyi Zhou · +10 more
Google XR Labs · University of California, Los Angeles · Carnegie Mellon University · University of Oxford · The University of Tokyo
Research Digest··2 min read
The authors introduce GenXR-Bench, the first benchmark for end-to-end evaluation of coding agents in extended reality (XR) software development.
§