Voice assistants that hold conversations must remember earlier turns and instructions. Their skills are usually tested on written transcripts of speech, not on the original audio. This makes it unclear if success on text means success on voice.
A was built covering ten languages. Six speech-language models were tested on both transcripts and direct audio of the same turns, comparing their item-by-item success on conversational requirements.
Transcript scores were an unreliable guide. The biggest failures on audio came from losing track of conversational history spoken earlier. Replacing the audio history with a text transcript sharply improved performance, showing that transcript testing hides critical audio processing failures, especially for maintaining context.