• D.C.
  • BXL
  • Lagos
  • Riyadh
  • Beijing
  • SG

Intelligence for the New World Economy

  • D.C.
  • BXL
  • Lagos
Semafor Logo
  • Riyadh
  • Beijing
  • SG


Media newsletter icon
From Semafor Media
In your inbox, 1x per week
Sign up

Exclusive / ‘Iterate through’: Why The Washington Post launched an error-ridden AI product

Max Tani
Max Tani
Media Editor, Semafor
Dec 14, 2025, 9:00pm EST
Media
Your Personal Podcast tool
The Washington Post’s “Your Personal Podcast” tool as seen on iOS.
PostEmailWhatsapp
Title icon

The Scoop

The Washington Post began putting out AI-generated podcasts even after internal tests found that the AI tech introduced errors and bias into the publication’s reporting.

More than two-thirds of scripts generated by the feature, dubbed “Your Personal Podcast,” failed a metric intended to determine whether they met the publication’s standards, according to a readout of the tests shared with Semafor.

“Testers were asked to rate the quality of scripts on a pass/fail basis (news used the categorization of publishable vs. not) in order to give us the most comprehensive list of issues to examine,” the company said in its internal review. The review added that when in doubt, testers told to fail scripts “as a precaution.” In three rounds of testing, between 68% and 84% of scripts failed.

Four Washington Post staff also described mistakes in personalized podcasts ranging from minor pronunciation issues to misattributed or fabricated quotes, as Semafor reported Thursday. The tool also sometimes inserts commentary, they said — for instance, by interpreting a source’s quotes as the paper’s position on an issue.

AD

The podcast tool’s prognosis was poor, the review concluded: “Further small prompt changes are unlikely to meaningfully improve outcomes without introducing more risk.”

Still, the company’s product review team recommended moving forward with the release, saying it would continue to “iterate through the remaining issues” with the newsroom and would label the tool as a work-in-progress that could generate errors.

The Post’s chief technology officer, Vineet Khosla, and head of product, Bailey Kattleman, announced the tool on Wednesday, saying in a note to staff that the podcasts are the “ultimate intersection across our critical initiatives of premium experiences, customer choice and AI platform and products.”

AD
Title icon

Know More

Executives across nearly every major industry are racing to develop AI-powered products, hoping to stay ahead of a wave of technological change.

News executives who’ve watched search traffic evaporate as their readers turn to chatbots are now simultaneously trying to cut deals with major AI players to license content to them; build their own AI tools to improve workflows; capture online interest; and sue AI companies that they believe have stolen their content to train their large language models.

Most major news organizations who use AI use it largely behind the scenes. As Semafor first reported earlier this year, The New York Times rolled out a suite of new tools for its product and editorial staff that could eventually write social copy, SEO headlines, and some code. But the company has bracketed its AI use, noting in internal documents seen by Semafor the potential risks for copyright infringement and exposure of sources. The Wall Street Journal and others have used AI to generate summaries of human-written stories, while Business Insider recently rolled out an AI tool that has generated a handful of stories edited by a human. Publications including The New Yorker use AI voices to read stories aloud.

AD

But in an industry whose currency has historically been trust, most news brands have been leery of turning their core relationship with their readers over to unreliable LLMs. The Post is the exception, racing to experiment with live tools. In February, the company launched Ask The Post AI, a bot that answers users’ questions about the news but can make basic errors. The company also plans to roll out an AI writing coach called Ember to readers who want to submit columns to its new opinion app, according to The New York Times.

Leaders on the editorial side of the paper expressed alarm about the AI podcast errors, which some in the newsroom said would be fireable offenses if made by a human journalist on staff. In a message to staff shared with Semafor, Post head of standards Karen Pensiero wrote that the mistakes have been “frustrating for all of us,” while another editor noted that it was a particularly poor time for the Post to produce shoddy content, given scrutiny from the Trump administration.

“It is truly astonishing that this was allowed to go forward at all,” one Post editor wrote in Slack messages shared with Semafor. “Never would I have imagined that the Washington Post would deliberately warp its own journalism and then push these errors out to our audience at scale. And just days after the White House put up a site dedicated to attacking journalists, most notably our own, including for stories with corrections or editors notes attached. If we were serious we would pull this tool immediately.”

Title icon

The View From The product side

The Post is far from the only company pushing into AI-generated personalized audio features.

Tech platforms like Google have rolled out audio summary tools designed to sound like podcasts. While many podcasters have scoffed at the idea, some popular podcasters are experimenting with AI to supplement their shows or plug holes.

The podcaster Stephen Bartlett used an AI clone of himself for a video series on CEOs, while the LA Times reported that a daily podcast host used an AI version of herself to fill in for her while she had laryngitis. Google’s NotebookLM podcast generator was one of the early breakout AI tools, and in the same week the Post rolled out its personalized podcasts, Yahoo released essentially the same AI-generated podcast product.

One senior product figure at a rival news organization said that the Post’s push into the product was justified given the types of products users say they want. “For all product teams, personalization often comes up as one of the most requested features in user research,” they said.

Title icon

Max’s view

The Post’s struggles with AI are in some ways a distraction from its basic crisis: a collapse in its subscriber base after owner Jeff Bezos shifted both his own politics and the paper’s toward Donald Trump. The Post is struggling to define an identity that blends its new credo, “Riveting Storytelling for All of America,” and a libertarian-ish opinion page with the #Resistance-era slogan “Democracy Dies in Darkness.”

As journalists and subscribers have moved to competitors like The Atlantic and anti-Trump digital media startup The Bulwark, the Post is attempting to iterate its way out of its subscriber and talent slump.

It’s not clear the new tools will actually help solve either problem. I struggle to envision a world in which a meaningful number of users go to a specific app for personalized podcasts — particularly when the primary problem with podcasts isn’t that there are too few of them.

But the Post’s decision to test out a bunch of products and see which ones work is a forward-thinking strategy for a paper that has too often found itself playing catch-up to its competitors, notably the Times, which has outperformed the paper in podcasts, games, cooking, events, and other offerings.

Title icon

The View From the Post

There’s some evidence to suggest that there is an audience for AI-generated podcasts. Inception Point, an AI podcast company that has generated 200,000 episodes of content featuring over 100 AI personalities, has garnered 400,000 subscribers across Apple and Spotify, according to the LA Times.

For the Post, Wednesday’s release is the first step to realizing a larger vision around products tailored to specific users.

“This is how products get built and developed in the digital age: ideation, research, design and prototyping, development, and then Beta. Only if they prove to be successful for the customer do they then get launched. As stated clearly on Your Personal Podcast, it is currently in Beta,” a Washington Post spokesperson told Semafor.

Title icon

Room for Disagreement

Personalization has long been the holy grail of digital news products, but even that goal has its skeptics.

“One of the main things we constantly find in our work is that news and personalization do not go together,” Craig Robertson, a postdoctoral research fellow at the Reuters Institute for the Study of Journalism told Semafor. “People don’t want to feel like they’re out of the loop. If my news is personalized, am I missing important stories? Am I not being told or exposed to certain things? I don’t want to be in a bubble.”

Title icon

Notable

  • The rest of the media business, from music to advertising, is also scrambling to adopt AI tools for personalization and scale.
  • Gina Chua, Semafor’s founding executive editor and a leading voice in AI in news, shared some more specific predictions: “Perhaps you’d get a shorter, audio version when your phone detects you’re driving or running; or one that omits background information it knows you read yesterday; or one that weaves in more context when the AI believes it’s a topic you’re unfamiliar with. Or a bot could rewrite the story using language at a level you’re comfortable with, or offer a tool that allows you to engage in an extended conversation to find out more,” she wrote.
  • The Post’s failure to produce a hit podcast to compete with The New York Times has frustrated the paper’s changing leadership. The company’s head of audio, Renita Jablonski, left last month to join The Bulwark.
  • Staff believe the company is likely to eventually integrate the company’s podcast and video offerings under one section as the two formats become increasingly intertwined.
AD
AD