The authors modify the Fixed-Point Reasoner (FPRM) framework to operate on the unit hypersphere, leveraging the normalized Transformer (nGPT) architecture.
Spherical geometry simplifies adaptive-loop reasoning and improves halting efficiency
By moving fixed-point computation to the unit hypersphere, the authors eliminate coupling in residual rescaling and achieve better accuracy and earlier halting on puzzle tasks.
Big Tech
Arber Zela · Artavazd Maranjyan · Pol Puigdemont · David Berghaus · Valentyn Boreiko · Volkan Cevher
EPFL · Lamarr Institute · Fraunhofer IAIS · Amazon AGI
Research Digest··2 min read
The authors introduce Spherical Fixed-Point Reasoners (SFPRM), which perform a damped fixed-point loop on the hypersphere of normalized transformers.
What it released
Code
§