Multimodal model shares latent number geometry, but in rotated coordinates

Causal interventions show text and image arithmetic rely on a common 13-dimensional numerical frame that is not visible to the digit readout.

Big Tech

Oriol Juan Sabater · Alex Ferrando de las Morenas · Pau Rodríguez · Xavier Suau

Universitat Autònoma de Barcelona · Apple

Research Digest·28 October 2026·3 min read

Illustration: Multimodal model shares latent number geometry, but in rotated coordinates

In one line

Multimodal models form an abstract, modality-independent numerical concept in a hidden subspace before reading out the answer digit by digit.

Before you read this

Three ideas this paper builds on

mechanistic interpretabilityThe study of how neural networks compute through internal circuits, features, and causal interventions, rather than just input-output behavior.94 papers →activation steeringActivation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.52 papers →counterfactual reasoningComparing actual outcomes with imagined alternatives to estimate the effects of different actions or conditions.82 papers →

Read it as

When an AI answers math from text and images, does it use the same internal idea of the number, or just produce the same output token?

Researchers found the exact internal state that controls the answer. Swapping this state between text and image versions of a problem tested if the representations are shared.

Direct swaps recovered only half the causal effect. A single geometric realignment recovered over ninety percent, proving one abstract concept exists. It lives hidden from the digits, so the model plans the entire answer before writing a single digit.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

mechanistic interpretability activation steering counterfactual reasoning vision-language model latent world model