Unmasking Synthetic Voices: Advances in Voice Cloning Detection

by Ralph Rodriguez, CPO
July 24, 2023

In today’s digital world, audio data is frequently used to validate users’ claimed identities. But bad actors, on the other hand, have been known to exploit bogus biometric data like synthetic speech signals to trick service providers into thinking they are legitimate users.

Spoofing is a type of impersonation (also known as a presentation attack) that poses a considerable problem to liveness detection and to the security of both organizations and their end users. Further research and collaboration are crucial to maintaining the integrity of digital identities amidst these evolving threats. In this spirit, Daon has created a novel method for detecting fake audio data, particularly in voice cloning scenarios, and our innovative Voice Cloning Detection (VCD) systems are constantly being improved upon to help our customers differentiate between genuine and fraudulent utterances.

Let’s investigate advances in synthetic speech identification algorithms and their capacity to detect voice clones, decrease training time, and lower costs involved with upgrading detection systems.

Understanding Voice Cloning Detection systems

Voice Cloning Detection systems are designed to differentiate between real and fake speech patterns, the latter of which are synthetically generated by voice cloning algorithms. To train these systems effectively, a training database containing both bona fide and fraudulent utterances is required. The accuracy of VCD systems improves when fraudulent utterances are generated by the same voice cloning algorithm used for training.

However, detecting fraudulent utterances generated by previously unencountered voice cloning algorithms poses challenges, as the results may fall short of the desired accuracy requirements needed by regulated industries to verify users calling in to their organizations’ contact centers.

Challenges in updating VCD systems

Updating VCD systems to detect fraudulent utterances from unexpected voice cloning methods generally entails getting the unknown algorithm, creating training data using that algorithm, and either constructing a new model or performing a whole re-training.

This process is complicated, time-consuming, and costly, especially given the growing rate at which new cloning algorithms are developed. The requirement for constant updates adds to the difficulties of maintaining good clone voice detection systems.

Ways to thwart voice cloning and enhance the detection of synthetic voices

As the development of voice cloning algorithms accelerates, staying ahead in the ongoing war against synthetic voice clones is critical. Synthetic speech algorithm improvements and their application in voice cloning detection systems constitute a big step forward in limiting the risks connected with fake audio data. However, it is critical that researchers, technology providers, and governments work closely together to further enhance and expand these detection approaches. We can improve our ability to recognize and counter the threats posed by voice cloning and preserve the integrity of audio-based authentication systems by encouraging innovation, sharing knowledge, and applying rigorous measures.

Key recommendations for enhancing synthetic voice detection include multi-factor authentication (MFA), robust liveness detection, dynamic voiceprints, continuous monitoring, and machine learning algorithms.

Multi-factor authentication (MFA)
Implementing MFA offers an added layer of security. By combining voice authentication with other authentication factors, such as passkeys and biometrics, you can create a security posture that makes it much more difficult for fraudsters to impersonate someone solely based on voice cloning. Note, however, that not all MFA systems are equal – and some offer both greater security and convenience for users over others.

Robust liveness detection
Utilize advanced liveness detection techniques to ensure that the voice being authenticated belongs to a live person and is not a pre-recorded or synthetic voice. Market-leading liveness detection can analyze various factors, like speech patterns, voice characteristics, and even physiological responses, to verify the authenticity of the speaker. Compliance with standards and independent validation of liveness capabilities are important to ensure efficacy.

Dynamic voiceprints
By employing dynamic voiceprints, you can capture unique characteristics of an individual’s voice in real-time. This tends to include analyzing factors such as pitch, rhythm, accent, and other speech patterns that are difficult to replicate by voice cloning algorithms. Regularly updating and adapting these voiceprints can make it harder for fraudsters to mimic a genuine user’s voice.

Continuous monitoring
Implement continuous monitoring during authentication sessions to detect any anomalies or suspicious patterns in the user’s voice. This can involve analyzing changes in speech characteristics, detecting unnatural pauses or glitches, or even comparing the current voice sample with previous samples to identify inconsistencies.

Machine learning algorithms
Leverage machine learning algorithms to continuously improve the accuracy and effectiveness of voice cloning detection systems. By training the algorithms with a diverse range of voice samples, including both genuine and synthetic voices, the algorithms can learn to more effectively differentiate between the two over time.

By combining these recommendations, organizations can enhance their defense against voice cloning attacks and ensure the integrity of their voice authentication processes. It is essential to remain vigilant, adapt to evolving threats, and employ a multi-faceted approach to protect against the ever-advancing techniques used in voice cloning.

Advances in synthetic speech algorithms

Using transfer learning for cloned voice identification techniques is a more recent approach created by Daon’s R&D team to solve the limits of existing VCD system update methods. This unique strategy promises to cut training time and upgrade costs. These developments enable more efficient training and deployment of VCD systems, ultimately improving their ability to detect fraudulent statements.

The implementation of this upgraded approach has various advantages. It decreases the time and costs associated with updating VCD systems, allowing enterprises to react to developing voice cloning algorithms more quickly. Rather than starting the retraining process from scratch, the new method allows for more targeted updates that use a limited quantity of fresh data to fine-tune the existing system. This tailored strategy conserves resources while streamlining the update process, resulting in a more rapid and cost-effective response to evolving voice cloning threats.

Introducing xSentinel, Daon’s cloned voice detection solution

xSentinel is a deepfake fraud indicator rather than a biometric authentication mechanism. This is an essential distinction; biometric data is categorized as special category data in many jurisdictions worldwide and its processing is more strictly regulated. Voice data is often classified as biometric data only if the system’s objective is to authenticate or identify the person. xSentinel does neither of these things; instead, it determines if the voice data was generated by a human or a synthetic voice generator.

How it works

xSentinel detects cloned voices in phone calls and alerts the agent that the caller may have utilized voice cloning technology. This is a strong fraud indicator that allows the agent to go above and beyond to validate the caller, as the sound of a human voice typically conveys information about the user’s origin (accent), gender, and age. While this information is imperfect, enterprises currently use it in voice-only conversation to provide context.

When a customer phones the contact center and identifies themselves, the representative usually has demographic information on hand. The agent intuitively compares this demographic information to the speech as a signal to detect probable fraud.

Now that deepfake speech generators are available, bad actors can imitate their targets’ accents, age, and gender, depriving the agents of the usefulness of that natural signal.

xSentinel removes the advantage that cloned speech generators give to fraudsters and offers an even stronger indication that potential fraud is taking place. Integrated into an enterprise contact center infrastructure, xSentinel can be deployed on-premises or in the cloud for use during a user’s interaction with an IVR or a contact center agent.

The path forward

The ability to detect synthetic voice clones is essential to the security and reliability of audio-based authentication transactions. The advancements in synthetic speech detection algorithms provide optimistic solutions to the difficulties associated with the modernization of VCD systems. By minimizing training time and expenses, enterprises can remain proactive in their efforts to combat the ever-changing landscape of voice cloning threats.

Continuing research, collaboration, and technological advancements will play a crucial role in bolstering our defenses against fraudulent audio data. By endeavoring to safeguard the integrity of audio-based authentication in a world that is becoming increasingly digital, we can enhance the veracity and dependability of digital identities.