• D.C.
  • BXL
  • Lagos
  • Dubai
  • Beijing
  • SG
rotating globe
  • D.C.
  • BXL
  • Lagos
Semafor Logo
  • Dubai
  • Beijing
  • SG


Newsletter͏‌  ͏‌  ͏‌  ͏‌  ͏‌  ͏‌ 
 
sunny Washington
thunderstorms Taipei
cloudy San Francisco
rotating globe
December 14, 2022
semafor

Technology

Technology
Sign up for our free newsletters
 
Reed Albergotti
Reed Albergotti

Hi, and welcome to Semafor Tech, a twice-weekly newsletter from Reed Albergotti and Louise Matsakis that gives an inside look at the struggle for the future of the tech industry. Today’s article started out as a simple attempt to write about how newfangled artificial intelligence tech behind ChatGPT is changing content moderation. It turns out that, for the most part, it’s not. The reason why turned out to be a much more interesting story, as you’ll see below.

Are you enjoying Semafor Tech? Help us spread the word!

Buy/Sell

➚ Buy: The U.S. dollar. The currency fell to a six-month low against the euro after federal data showed inflation has begun easing. Some tech companies previously blamed their lower-than-expected earnings on foreign exchange costs. That should be less of a problem soon.

➘ Sell: Crypto. Sam Bankman-Fried, the disgraced founder of the crypto exchange FTX, has been arrested in the Bahamas and is facing an extradition request to the U.S. Meanwhile, Korean authorities said that Do Kwon, another wanted crypto founder, is on the run in Serbia. (Bankman-Fried is an investor in Semafor.)

TweetEmail
Semafor Stat

The factor by which searches for terms related to COVID-19, including “N95 masks” and “ibuprofen” have increased in China over the past week, according to the Chinese search engine Baidu. Last month, rare protests against Covid restrictions broke out across the People’s Republic. The government subsequently loosened many of them, leading to a sharp rise in infections.

TweetEmail
Reed Albergotti

Breakthrough AI tools still rely on old-school moderation techniques

Unsplash - Chris Schramm

THE SCOOP

The breakthrough artificial intelligence technologies that have captivated the internet in recent months still rely on outside help, including human beings, for one task: Content moderation, according to people familiar with the matter.

OpenAI, for instance, which makes ChatGPT and the image creation tool DALL-E, uses a blend of internal and external people for content monitoring, according to a person familiar with the matter.

The fact that these highly advanced, breakthrough technologies still require scanning with automated tools that use older software shows how these otherwise impressive applications have some significant limitations.

While the “generative AI” technology behind ChatGPT, DALL-E and other products, is able to wow users by drawing on a vast amount of internet data, its ability to truly understand what it’s creating and the concept of content moderation, is still a work in progress.

Products like ChatGPT, for instance, can answer almost any question in complete sentences that are indistinguishable from a human response. It can also write software code. Products like DALL-E and Stable Diffusion from Stability AI are able to conjure up original works of art from a simple text description.

Kevin Guo, co-founder and CEO of Hive, a content moderation firm whose clients include BeReal and Reddit, said he recently ran an experiment testing generative AI for content moderation. The test compared the new technology to “deep learning,” which was until recently the most advanced AI technology.

“The traditional model dramatically outperformed,” he said. “We’re talking multiples more in accuracy.”

OpenAI is working on its own content moderation tools using generative AI, according to an August blog post on the company’s site. In research papers, OpenAI has also described using human contractors to help identify data that is offensive.

REED’S VIEW

In every big breakthrough in artificial intelligence, from self-driving cars to releasing ChatGPT, a media hype cycle prompts warnings about the rise of intelligent machines.

Then, there’s a letdown when the technology’s limits become clear.

The outside assistance used by OpenAI represents a technological boundary of sorts, and an important new clue about what these novel forms of AI have learned and what they have not.

Products like DALL-E, ChatGPT and Stable Diffusion do represent major advances in technology that allow us to do things that were impossible just a few years ago.

The truly useful mass market AI tools will no doubt change entire industries as more people figure out how to put it to use.

Guo said the thing that makes generative AI so impressive — its ability to draw on massive amounts of written language — also makes it struggle with specific tasks, like determining whether a piece of content violates policies.

“You’ll see examples where it’s very magical. On the other hand it can seem kind of stupid,” Guo said. “What are we seeing here? Is it true intelligence or is it a massive mimicry engine that fails the process if you give it something it hasn’t seen before?

And there’s another problem for generative AI: Cost. Because the models use such vast pools of data, they consume more processing power for each task. Sam Altman, CEO of OpenAI estimated it costs “single-digit cents” every time someone asks ChatGPT a question. “The compute costs are eye-watering,” he tweeted.

Guo said using generative AI to do content moderation would increase costs by roughly 100 times. (Right now, he said Hive charges around 25 cents for 1,000 pieces of content it scans for a company.)

The high costs would make generative AI prohibitively expensive for companies like Google and Facebook, which have billions of users.

Artificial intelligence researchers at Microsoft did find a way to use generative AI in content moderation: By asking it to make more offensive content to help put traditional content moderation tools to the test.

In a paper, authored by Hamid Palangi, researchers described a new tool called “ToxiGen” that acted like a digital “red team,” using OpenAI’s GPT-3 “large language model” to try and find holes in content moderation tools so that they could later be plugged.

Ece Kamar, an AI researcher at Microsoft Research said it would be some time before technologies like large language models make sense as content moderation tools themselves, mainly because of high costs and low success rates at catching prohibited content.

“These technologies are getting more powerful, but at the same time, we are all learning what they’re good at, what they’re not good at, and how they are going to get better,” she said. “I think we are always gonna have humans in the loop.”

ROOM FOR DISAGREEMENT (from ChatGPT)

While it is true that generative AI technology may have difficulty understanding complex concepts like content moderation policies, it is important to note that these technologies are still in their early stages of development. It is not fair to say that generative AI is not capable of performing content moderation tasks, especially when it has shown the ability to generate human-like responses and original works of art.

Additionally, it is worth noting that the results of a single experiment performed by a single company do not necessarily reflect the capabilities of generative AI as a whole. As the technology continues to advance, it is likely that its ability to understand and implement complex concepts will improve – Copied and pasted directly from ChatGPT.

THE VIEW FROM CHINA

In the West, China is often depicted as a technological dystopia, where artificial intelligence is used to swiftly censor all forms of dissent on the internet. But like their Western counterparts, Chinese tech companies still rely on vast amounts of human labor to scrub social media of offending content. Earlier this year, some of these workers spoke to the news site Sixth Tone, who reported “their job required them to monitor online content at all hours of the day, in shifts that sometimes stretched up to 14 to 15 hours.”

Louise

TweetEmail
Text

One Good Text with... Harini Janakiraman

TweetEmail
China Window

During the opening ceremony for TSMC’s new plant in Arizona last week, Morris Chang, founder of the legendary Taiwanese semiconductor company, made a sobering speech. “Globalization is almost dead,” he said. “Free trade is almost dead. And a lot of people still wish they would come back, but I really don’t think they will be back for a while.” Chang’s words didn’t attract much attention in U.S. media, as Kevin Xu noted in his newsletter, Interconnected.

But they should have, because Chang was talking bluntly about a seismic shift in the global economy. As the 91-year-old has pointed out many times, it doesn’t make economic sense for TSMC to manufacture chips in a place like Arizona, where costs are significantly higher. But since the relationship between the U.S. and China has deteriorated, companies like TSMC now need to prioritize politics and security just as much, if not more, than their bottom lines.

— Louise

TweetEmail
Evidence

TweetEmail
Staff Picks
  • Elon Musk’s appearance on stage at a Dave Chappelle comedy performance in San Francisco was stymied by a booing crowd. There’s some disagreement on the booing-to-cheering ratio, but both Chapelle and Musk seem to have been caught off guard. Musk has used his megaphone as owner of Twitter to deride the LGBTQ+ community and amplify conservative causes, so it’s no surprise he’d meet detractors in San Francisco. What’s stunning is that Musk’s reputation as a right-wing agitator is reaching parity with his reputation as perhaps the greatest entrepreneur of his generation. Tesla’s stock is down almost 60% this year and the electric vehicle maker is on its way to becoming a “partisan brand.” It will be fascinating to see how Musk is ultimately remembered in history.
  • The story of human progress is often told through the lives of inventors and scientists who make the awe-inspiring discoveries that change the world. But a new feature in The Atlantic makes a compelling case for why the “eureka moment” is an unhelpful myth. In reality, the creation of new technology is just the beginning. It takes a village to implement it, and as the article argues, America’s individualist society often stands in the way of fully benefiting from its own inventions, like solar cells, semiconductors, and nuclear energy.
  • Is Apple finally giving in on its walled garden? A story in Bloomberg says Apple will begin allowing app stores other than its own on its devices. The change will take place only in Europe, where regulators have been cracking down on Apple’s vice grip on the mobile software market. The change could allow new businesses to flourish on mobile, but let’s not get ahead of ourselves. Apple will look for ways to make alternatives less attractive than its own app store.
TweetEmail
Enthusiasms

Dr. Marvin Adams of the Department of Energy
Reuters/Mary F. Calvert

The U.S. Department of Energy announced a major breakthrough Tuesday in nuclear fusion, creating a reaction that produced more energy than it consumed. Fusion is better than Homer Simpson-style fission for three main reasons: It doesn’t create long-lasting nuclear waste, so no need to worry about three-eyed fish. It doesn’t use fissile materials like uranium and plutonium, so people are less likely to try to steal it to make weapons (The reason Marty McFly ended up in 1955). And it won’t melt down, like in Fukushima.

Fusion will surely play a role in our future energy grid. How long it takes depends on how much taxpayer money and private investment goes into the research. Fusion reactions still need to get better before they’re cost-effective, and scientists need to figure out how to build equipment that can withstand the extreme forces of fusion — it’s four times hotter than the Homer Simpson version.

— Reed

TweetEmail
How Are We Doing?

Are you enjoying Semafor Tech? The more people read us, the better we’ll get. So please share it with your family, friends and colleagues to get those network effects rolling.

And hey, we can’t inform you on what’s happening in tech from inside your spam folder. Be sure to add reed.albergotti@semafor.com (you can always reach me by replying to these emails) and lmatsakis@semafor.com to your contacts. In Gmail, drag this newsletter over to your ‘Primary’ tab.

Thanks for reading.

Want more Semafor? Explore all our newsletters at semafor.com/newsletters

— Reed and Louise

TweetEmail