MoLE is a framework for latent visual reasoning that forces different latent tokens to attend to different visual evidence and transform it differently. It uses sparse visual routing to assign each visual token to a subset of latent visual experts, an expert-specific value adapter to specialize the transformation, and latent summary experts to aggregate the complementary states. Across five visual reasoning benchmarks, MoLE averages 78.6, beating data-matched supervised fine-tuning by 4.9 and the strongest latent visual reasoning baseline at the same latent budget by 3.6. Latent states are less similar to each other and their visual attention patterns are more diverse. Removing the latent pathway at inference drops average performance by 9.2, showing the latents carry information the model actually uses. These results matter because they indicate the bottleneck in latent visual reasoning is not the number of latent tokens but how complementary they are. Specialization can improve a fixed budget without requiring explicit reasoning traces or intermediate targets.
Specializing latent tokens as visual experts improves vision-language reasoning
MoLE forces latent tokens to extract complementary visual evidence, boosting accuracy across five benchmarks without predefined roles
Tongji University · Hong Kong Polytechnic University · Alibaba Group · University of Electronic Science and Technology of China
In one line
In latent visual reasoning, specializing latent experts to complementary evidence improves performance more than increasing the number of latent tokens.
Before you read this
Two ideas this paper builds onMoLE applies sparse visual routing per decoder layer to assign visual tokens to subsets of latent visual experts, and uses Expert-Specific Value Adapters (ESVAs) to transform evidence differently.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.