Audio carries spatial cues like direction and distance through differences in timing and loudness between microphones, but existing audio AI models discard that information by converting everything to a single channel to keep performance stable. This means they cannot answer questions about where a sound is coming from or how it moves.
The researchers built SpheraAudio, which accepts recordings with any number of channels, from a single mic to an array that captures full 3D sound fields, and converts them into a single stream of tokens the language model can use. A special spatial encoder extracts location cues and inserts them at the right moments in the audio stream, while a custom position encoding keeps track of which tokens are spatial and which are plain audio.
SpheraAudio achieved the best scores on existing spatial audio benchmarks and a new test they created, which includes questions about a single sound's direction and distance as well as multi-step reasoning over multiple sounds. Crucially, it maintained its general audio understanding, proving that spatial awareness does not come at the cost of other capabilities.
