TikTok’s AI tool accidentally let you put Hitler’s words in the mouth of a paid actor

TikTok accidentally posted a link to an internal version of its new AI digital avatar tool with no safeguards, allowing users to create videos that say anything. Hiccup was first spotted by CNN and allowed outlets to create videos containing quotes from Hitler and messages telling people to drink bleach, among other phrases. TikTok has since removed this version of the tool, while the version TikTok launched is still available.

launched earlier this weekof TikTok Symphony Digital Avatar Allows businesses to create ads using the likeness of paid actors. It also uses AI-powered dubbing that lets advertisers enter a script to make avatars say what they want, within TikTok’s guidelines. Even though only users with a TikTok Ads Manager account can use this tool, the version CNN found allows anyone with a personal account to try it out.

In a statement VergeTikTok spokesperson Laura Perez says TikTok has resolved a “technical error” that “allowed a very small number of users to create content using an internal test version of the tool for a few days.”

When CNN discovered the internal tool, it allowed the outlet to create videos featuring Osama bin Laden’s “Letter to America,” a white supremacy slogan, and videos telling people to vote on the wrong day. None of the videos CNN created had a watermark indicating the video was generated by AI, a fair version of TikTok’s Symfony digital avatar.

CNN did not post the video it created on TikTok, but Perez said that if it had, the content “would have been rejected for violating our policies.” Though TikTok has removed this version of its tool, it raises the question of whether people will find other ways to abuse the digital avatar creator — and whether TikTok is prepared for that.