Voice security systems treat a voice as proof of who someone is, checking recordings against the speaker's identity label. But people do not always hear things that way, and modern synthetic speech can copy a voice so closely that listeners cannot tell it is fake.
To measure this, listeners were played pairs of recordings and asked whether both were the same person. The pairs mixed real recordings, synthetic copies (voice clones, made by a text-to-speech system that generates speech from text), and blends of two speakers' voices. Eleven speech models were then scored on how well their similarity judgments matched the listeners' answers.
Listener judgments went against the identity label most often for clones, where 41 percent of responses called a clone and a real recording of the same speaker different people. No model reached the level of agreement that listeners show with each other, so current systems do not yet capture how people actually hear identity.