What they did

The authors created CUA-Universe around three components. App-Forge packages real applications into reproducible virtual machines and discovers, wraps, or generates command-line interfaces; the resulting environments cover 16 applications across multiple domains.

Task-Weave constructs tasks of adjustable difficulty by combining reusable operations with seed files. Path-Steer then guides agent rollouts toward efficient GUI-and-CLI paths, verifies the resulting trajectories, and collects them as post-training data for a 9B computer-use model.

Key findings

• On CUA-Verse, the trained model gained 39.3 score points while using 37% fewer steps and 60% fewer tokens.
• On OSWorld, success rate increased by 16.8 percentage points, with 57% fewer steps and 44% fewer tokens.
• On OSWorld-MCP, score rose by 7.84 points while steps fell 27% and token use fell 30%.
• The authors report a behavioral shift away from GUI-only interaction and brittle command-line scripts toward coordinated use of both interfaces.

Why it matters

Many computer tasks require visual interpretation of interface state but are faster and more precise when bulk operations are executed through commands. CUA-Universe offers a way to produce hybrid training environments and verified trajectories without separately hand-engineering every task, suggesting that interface selection itself can be learned as part of an agent's control strategy.

Caveats

The reported environment covers 16 applications, so its scalability to substantially broader and less controlled software remains untested here. The abstract does not specify baseline models, statistical uncertainty, per-application performance, security constraints around command execution, or how well the trained behavior transfers beyond the three named evaluations.