AI companies listen to voice recordings

When the boss calls with a deepfake voice

Artificial intelligence not only fakes faces in videos. Voices can also be cloned with it. Apparently someone wanted to bring a lot of money into a tech company with one of these. Here you can listen to the fake voice.

From Michael Förtsch

They are still rarely heard: voices from real people, which are imitated by an artificial intelligence. Some YouTubers have made it an art, for example replicating the voice of Donald Trump, Adam Driver or Joe Biden and then having them say absurd or funny things. A lot of work and computing time on a computer are behind such audio deepfakes. Sometimes it takes weeks, for example, the YouTuber and AI hobbyist who runs the channel told us Speaking of AI and has created some of the most compelling audio deepfakes to date. But voices can also be duplicated much faster - but with audibly worse results. Nevertheless, according to the IT security company NISOS, these are apparently increasingly used. For attempted fraud.

In June, the security company reported, an employee of a technology company was said to have received a bizarre voice message from someone posing as the company's boss. He has requested "immediate assistance in closing an important deal" and instructed to forward a payment. At least it sounded like it. However, the clerk was skeptical and did not pass on the referral that the voice requested, but informed the legal department. As Rob Volkert of NISOS writes, "[the corresponding voice] definitely sounds human," but quite noisy and distorted even for a poor phone connection.

Become a member of 1E9 - for as little as 3 euros a month!

As a member, you support independent, future-oriented tech journalism that is created for and with a community of idealists, founders, nerds, scientists and creatives. You also get full access to the 1E9 community, exclusive newsletters and can be part of 1E9 events.

Join Now!

When examining the voice message, from which NISOS released a short sound snippet, it would have been noticed that the audio components were "chopped off" and strung together inconsistently. Background noises could not be assigned to a natural source such as a busy place or road traffic, but were evidently artifacts of the audio synthesis. NISOS could not determine with which software or which language AI the deepfake was created. This would have required significantly more audio material.

If only the boss sounds almost like the boss

Basically, it is no longer difficult to create an audio deepfake. With deep voice and real-time voice cloning, there is free software on the Internet that, with a little training, allows you to clone your own voice or someone else’s voice with short audio snippets - at least for English voice messages, because the AI ​​models are trained on this. We at 1E9 played around with it ourselves - as you can hear from the language example in which Elon Musk admits that he is "of course an alien". Start-ups such as Resemble.ai, Lovo.ai or Replica Studios also offer voice cloning as a paid service. Resemble.ai, for example, wants to prevent "malicious use" of its service by uploading a video-recorded consent if someone else's voice is not to be cloned.

twitter.com

Michael | Förtsch (Micha)

As NISOS explains in its report, there are several reasons why the audio deepfake was not convincing. Or rather, there are several things that would-be scammers should have done better. Among other things, they should have trained the artificial intelligence with more and clearer voice recordings from the company boss in order to generate an even more human and less artificial voice. In addition, the content of the voice message was too obscure. There was no clear context and no explanation of the circumstances that were decisive for the allegedly so urgent transfer.

In fact, other criminals had done better in the past. As recently as February, the IT security company Symantec reported three cases in which the voices of successful company founders had apparently been cloned using voice recordings from YouTube, social media channels or lectures such as the TED Talks. These were then used to try to bring the company into the millions - with Symantec leaving it open whether that was successful or not. In any case, one attack from last year was successful. A British manager was instructed by a fake voice to transfer 220,000 euros to a Hungarian account. He thought he had the head of the German parent company on the phone and did as he was told.

Teaser image: Getty Images / oxygen