Microsoft unveils real-time voice AI models for faster agent interactions

New MAI models offer streaming transcription and expressive text-to-speech, but latency and turn-taking challenges remain

By LineZotpaper

Published23 October 2026

Read Time2 min

Microsoft has released three new voice-centric AI models through its Microsoft Foundry platform, designed to reduce awkward pauses in human-machine conversations. The models, announced on October 1, include a streaming transcription service that produces hypotheses within milliseconds of receiving audio, allowing applications to act on spoken words before a speaker finishes.

The new models include MAI-Transcribe-2-Streaming, which turns live speech into text incrementally, and two text-to-speech models: MAI-Voice-2.1 for expressive multilingual output and MAI-Voice-2.1-Flash for responsive, high-volume applications. Naomi Moneypenny, senior director of product development at Microsoft Foundry, wrote that live transcription can surface words almost as quickly as they are spoken, enabling customer-service agents to identify a caller's request before the sentence is complete.

Industry observers see potential but caution about complexities. Will Bodewes, CEO of voice-agent platform Phonely, noted that Microsoft's entry signals that voice AI is ready for prime time, but warned that the model is only the beginning. "Voice agents need to learn from every call and measurably improve over time," he said. Bodewes added that in the early days of cloud-based speech APIs, a speech request could sit behind a thousand others for 30 seconds, which on a phone call is a hang-up. Phonely built routing infrastructure to achieve 120-millisecond responses.

Amit Kiran Rege, a machine learning researcher at Phonely, highlighted the challenge of turn-taking. "The hard part is the silence in the middle where the model is deciding whether a caller is done or taking a breath," he said. His team found that speeding up the data pipeline made the agent interrupt callers more: two-thirds of the time the agent decided to jump in too early, and when latency was halved, those became audible interruptions. "The lesson is that latency and turn-taking have to improve together," Rege cautioned.

Microsoft positions the models for customer-service agents, multilingual assistants, interactive learning experiences, and live captions. According to Moneypenny, MAI-Transcribe-2-Streaming debuted at number one for accuracy on the Artificial Analysis leaderboard for both partial and final transcripts.

Analysis

Why This Matters

Voice AI could significantly improve customer service and virtual assistant interactions by reducing lag and enabling faster responses.
The technology promises to make automated call handling more natural, but implementation must carefully manage conversational flow to avoid unnatural interruptions.
Microsoft's entry with dedicated models may accelerate adoption across contact centers and voice-driven applications.

Background

Microsoft has offered Azure Speech services for years, and the new MAI models are part of the Microsoft Foundry platform for building and deploying AI agents. The streaming transcription model builds on the field of incremental speech recognition, where partial hypotheses are refined as more audio arrives. Competitors like Google, Amazon, and OpenAI also offer speech-to-text and text-to-speech services, but Microsoft is emphasizing low-latency turn-taking suitable for live conversations.

Key Perspectives

Microsoft (Naomi Moneypenny): The new models deliver industry-leading accuracy and speed, with partial transcriptions available within hundreds of milliseconds. This enables agents to act before a caller finishes speaking. Phonely (Will Bodewes and Amit Kiran Rege): Voice AI is ready for prime time, but the model alone is insufficient. Systems must learn from calls, enforce critical steps in code rather than leaving them to the LLM, and balance latency with turn-taking to avoid premature interruptions. Critics/Skeptics: While Microsoft's benchmarks are strong, real-world deployment in millions of calls reveals that reducing latency can increase interruptions. Developers must carefully tune turn-taking logic alongside speed improvements.

What to Watch

Adoption rates of MAI-Transcribe-2-Streaming in customer-service call centers.
Independent benchmark comparisons on the Artificial Analysis leaderboard.
Competitor responses from Google, Amazon, and startups building voice agents.

Sources

Microsoft’s new voice AI can start working out what you want before you finish speaking · The New Stack

Checked

10 claims in this story checked against 1 source: 6 supported.

Show each claim

Partly supportedMicrosoft unveiled real-time voice AI models for faster agent interactions.The New Stack: “A voice assistant can start reasoning or preparing a tool call sooner.”
Partly supportedMicrosoft released three voice-centric models designed to reduce awkward pauses.The New Stack: “A delay at either end can turn a conversation into a series of awkward pauses”
SupportedThe models were announced on October 1.The New Stack: “Naomi Moneypenny , senior director of product development at Microsoft Foundry Models division, blogged on October 1 to introduce three new Microsoft AI (MAI) models”
Partly supportedThe transcription service produces hypotheses within milliseconds of receiving audio.The New Stack: “produces its first hypotheses (known as partials, associated with the field of incremental speech recognition) within the “low hundreds of milliseconds” of receiving audio.”
SupportedMoneypenny said live transcription can surface words almost as quickly as they are spoken.The New Stack: ““A live transcription experience can surface words almost as quickly as they are spoken.””
SupportedBodewes said voice agents need to learn from every call and measurably improve over time.The New Stack: “Voice agents need to learn from every call and measurably improve over time.”
Partly supportedBodewes said a speech request could wait behind a thousand others for 30 seconds, and Phonely built routing infrastructure to achieve 120-millisecond responses.The New Stack: ““We built routing infrastructure just to find the least-loaded endpoints. Owning the model meant owning the queue, and that’s how we got to 120-millisecond responses,” explains Bodewes.”
SupportedRege said the hard part is the silence while the model decides whether a caller is done or taking a breath.The New Stack: “The hard part is the silence in the middle where the model is deciding whether a caller is done or taking a breath before the next word”
SupportedRege said the agent jumped in too early two-thirds of the time, and halving latency made those interruptions audible.The New Stack: ““Two-thirds of the time our agent decided to jump in too early, and nobody heard it, because the caller resumed before the first syllable and we canceled the reply. Cut your latency in half, and those become audible interruptions.””
SupportedMoneypenny said the transcription model debuted at number one for accuracy on partial and final transcripts on the Artificial Analysis leaderboard.The New Stack: “Her post noted that MAI-Transcribe-2-Streaming “debuts at #1 for accuracy” on both partial and final transcripts on the benchmark-based Artificial Analysis leaderboard.”