Semafor Signals
Supported by
TikTok’s AI watermarks could help curb deepfakes, but it’s no panacea
Insights from Bulletin of the Atomic Scientists, 404 Media, and The Japan Times
The News
TIkTok will start using a technology that detects whether an uploaded photo or video was made using another app’s artificial intelligence tools, the company said Thursday.
The tech is a kind of digital watermark called “Content Credentials,” which was initially spearheaded by Adobe and later embraced by other companies, including OpenAI. Watermarks on images generated by OpenAI’s app, Dall-E, for example, will appear if those images are uploaded to TikTok.
TikTok encourages and in some cases requires creators to label AI-generated content, but the new system is specifically designed to detect when AI content made by other apps makes its way onto TikTok.
YouTube, Instagram, and Facebook have also said they will begin using Content Credentials, but experts have questioned the efficacy given that AI is getting harder to detect and has a track record of fooling people and platforms.
SIGNALS
Watermarks are well-intentioned, but not always effective
Big Tech companies see watermarks as a promising tool for distinguishing between human-made and AI-generated content, and US President Joe Biden recently ordered the Department of Commerce to develop guidance for watermarking. While watermarking is “well-intentioned,” and a “step in the right direction,” it shouldn’t be relied on to address deepfakes, a British tech lobbying executive told NBC News. Researchers “have demonstrated over and over again that every single watermarking technique is vulnerable to exploitation and evasion,” the Bulletin of the Atomic Scientists wrote: Screen-shotting, cropping, or using a phone to take a photo of a piece of watermarked content is an easy way to bypass the controls.
Big Tech firms haven’t eradicated deepfakes
AI photos are rampant on social media despite companies’ assurances that they moderate and remove such misleading content from their platforms. Facebook’s own recommendation algorithm is promoting AI-generated images run by scammers — such as Jesus portrayed as a shrimp — Stanford and Georgetown University researchers found. “Because AI-generated spam works, increasingly outlandish things are going viral and are then being recommended to the people who interact with them,” 404 Media wrote. During the Met Gala on Monday, a fake red carpet image of Katy Perry, who did not attend, went viral, tricking her own mother, as well as Instagram’s own official account, which commented on the photo. The photo has since been labeled as “altered” on Instagram.
Japan not tricked by deepfakes
Responding to a crush of AI-generated content during India’s weekslong election, the country’s Election Commission recently warned political parties that they have three hours to remove deepfake content they promoted, once it’s brought to their attention. In Japan, meanwhile, residents aren’t easily fooled by misinformation, The Japan Times wrote. The country still relies on and trusts traditional media, and the internet is generally used for personal purposes like shopping and gaming — not discussing news or politics. The Japanese language also makes it harder for foreign propagandists to spread convincing lies.
