ElevenLabs’ AI voice generation ‘very likely’ used in a Russian influence operation

Date:

Share post:


Generative AI has a plethora of well-documented misuses, from making up academic papers to copying artists. And now, it appears to be cropping up in state influence operations.

One recent campaign was “very likely” helped by commercial AI voice generation products, including tech publicly released by the hot startup ElevenLabs, according to a recent report from Massachusetts-based threat intelligence company Recorded Future.

The report describes a Russian-tied campaign designed to undermine Europe’s support for Ukraine, dubbed “Operation Undercut,” that prominently used AI-generated voiceovers on fake or misleading “news” videos.

The videos, which targeted European audiences, attacked Ukrainian politicians as corrupt or questioned the usefulness of military aid to Ukraine, among other themes. For example, one video touted that “even jammers can’t save American Abrams tanks,” referring to devices used by US tanks to deflect incoming missiles – reinforcing the point that sending high-tech armor to Ukraine is pointless.

The report states that the video creators “very likely” used voice-generated AI, including ElevenLabs tech, to make their content appear more legitimate. To verify this, Recorded Future’s researchers submitted the clips to ElevenLabs’ own AI Speech Classifier, which provides the ability for anyone to “detect whether an audio clip was created using ElevenLabs,” and got a match. 

ElevenLabs did not respond to requests for comment. Although Recorded Future noted the likely use of several commercial AI voice generation tools, it did not name any others besides ElevenLabs.

The usefulness of AI voice generation was inadvertently showcased by the influence campaign’s own orchestrators, who – rather sloppily – released some videos with real human voiceovers that had “a discernible Russian accent.” In contrast, the AI-generated voiceovers spoke in multiple European languages like English, French, German, and Polish, with no foreign-soundings accents.

According to Recorded Future, AI also allowed for the misleading clips to be quickly released in multiple languages spoken in Europe like English, German, French, Polish, and Turkish (incidentally, all languages supported by ElevenLabs.)

Recorded Future attributed the activity to the Social Design Agency, a Russia-based organization that the U.S. government sanctioned this March for running “ a network of over 60 websites that impersonated genuine news organizations in Europe, then used bogus social media accounts to amplify the misleading content of the spoofed websites.” All this was done “on behalf of the Government of the Russian Federation,” the U.S. State Department said at the time.

The overall impact of the campaign on public opinion in Europe was minimal, Recorded Future concluded.

This isn’t the first time ElevenLabs’ products have been singled out for alleged misuse.  The company’s tech was behind a robocall impersonating President Joe Biden that urged voters not to go out and vote during a primary election in January 2024, a voice fraud detection company concluded, according to Bloomberg. In response, ElevenLabs said it released new safety features like automatically blocking voices of politicians.

ElevenLabs bans “unauthorized, harmful, or deceptive impersonation” and says it uses various tools to enforce this, such as both automated and human moderation. 

ElevenLabs has experienced explosive growth since its founding in 2022. It recently grew ARR to $80 million from $25 million less than a year earlier, and may soon be valued at $3 billion, TechCrunch previously reported. Its investors include Andreessen Horowitz and former Github CEO Nat Friedman. 



Source link

Lisa Holden
Lisa Holden
Lisa Holden is a news writer for LinkDaddy News. She writes health, sport, tech, and more. Some of her favorite topics include the latest trends in fitness and wellness, the best ways to use technology to improve your life, and the latest developments in medical research.

Recent posts

Related articles

Google is replacing Google Assistant with Gemini

Google will replace Google Assistant on Android phones with Gemini later this year, the company announced on...

Testing the Uber-Waymo robotaxi, Rivian goes hands-free, and Travis Kalanick has AV FOMO 

Welcome back to TechCrunch Mobility — your central hub for news and insights on the future of...

Tern AI’s low-cost GPS alternative actually works

We’ve all experienced that moment of frustration when the GPS glitches and you miss an exit on...

China is reportedly keeping DeepSeek under close watch

China appears to think homegrown AI startup DeepSeek could become a notable tech success story for the...

iPhone and Android users will soon be able to send encrypted RCS messages to each other

Text messages sent between iPhones and Android devices will soon benefit from end-to-end encryption (E2EE), after the...

Developer of Lockbit ransomware gets extradited to the United States

Rostislav Panev, a 51-year-old dual Russian-Israeli national, who is accused of being a key developer for the...

US lawmakers urge UK spy court to hold Apple ‘backdoor’ secret hearing in public

A group of bipartisan U.S. lawmakers are urging the head of the U.K.’s surveillance court to hold...

Kerry Washington invests in wedding marketplace Cheersy

Kerry Washington is expanding her angel investment portfolio, serving as lead investor in the pre-seed round of...