The paper presents MintAct, a family of that handle , multi-step navigation across mobile, desktop, and web, and within a single model. The models are trained at 2B, 4B, and 8B scales using a multi-stage recipe: supervised fine-tuning with both high-resolution single-step and low-resolution multi-step data, rejection sampling fine-tuning, and . An asynchronous RL infrastructure manages heterogeneous environments and long multimodal trajectories while controlling the cross-domain training distribution. The unified model matches or exceeds per-domain specialists on benchmarks like OSWorld-Verified (48.9), Online-Mind2Web (39.1), and AndroidWorld (67.0). This shows that unification does not sacrifice per-domain performance, which is important for reducing deployment cost and enabling on-device use.
A single visual agent spans mobile, desktop, web, and tool use
MintAct combines interface grounding and multi-step action across several digital environments while matching similarly sized domain-specific agents.
Big Tech
Mingfei Gao · Rui Tian · Haiming Gang · Bohan Zhai · Le Zhang · Yuanzheng Gong · +8 more
Apple
Research Digest··2 min read
In one line
MintAct is a family of vision-language models that unifies UI grounding, multi-step navigation, and visual tool use across mobile, desktop, and web at 2B-8B scales.
Before you read this
Three ideas this paper builds onvision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.251 papers →agentic reinforcement learningTraining approach that lets LLM-based agents learn multi-step tool use and planning via reinforcement learning, often with dense or per-tool rewards.120 papers →ui groundingMapping natural language commands or descriptions to specific visual elements of a user interface.24 papers →
Read it as
Gao and colleagues trained MintAct vision-language agents at 2B, 4B, and 8B parameter scales using infrastructure that runs hundreds of heterogeneous digital environments concurrently.
The authors developed one model family for —locating interface elements from visual input—multi-step navigation on mobile, desktop, and web platforms, and .
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.