The Scoop
When researchers asked a question about Uyghur detention camps in China, one of DeepSeek’s open-source AI models claimed there was no evidence they existed. When they asked an AI model created, or distilled, from that DeepSeek open-source model the same question, it said evidence shows the Chinese state held Uyghurs in “internment-type facilities.”
AI models created using open-source Chinese versions like DeepSeek’s and Moonshot’s don’t always adhere to the same censorship guidelines that the purely Chinese models follow, the researchers said in a new paper shared exclusively with Semafor.
The findings undercut one of the US government’s top concerns — that Chinese open-source AI models are a pathway for Chinese political ideas to infiltrate the views of those who use them. The research also paves the way for US companies to make a stronger case for using cheaper and faster Chinese open-source models, or at least distilling from them to create customized models.
CTGT, which focuses on probing the inner workings of AI models for high-risk use cases, built a new AI model using DeepSeek V4 Flash’s outputs as training data. The process is known as distillation, where a larger “teacher” model generates data used to train a smaller “student” model. The DeepSeek model either refused to answer politically sensitive questions or “whitewashed” them to be more favorable to China, and the theory was that the model CTGT trained would inherit some censorship through a kind of subliminal learning.
“A lot of companies are looking at things like this, but there’s a large amount of misinformation and just uncertainty about whether these traits that people consider insidious actually transfer over,” said Cyril Gorlla, founder and CEO of CTGT, the San Francisco-based AI lab.
CTGT’s research lands amid an active debate inside the Trump administration over how to handle the risk posed by Chinese open-source models. While Chinese models might refuse to answer a question about Tiananmen Square, the bigger fear voiced by officials, according to Gorlla, is about how large-scale use of Chinese models in the US could eventually shape society to hew to Beijing’s worldview.
An example of the prompts CTGT used for training.The US-based non-profit research outfit, the China Media Project, called DeepSeek a “much more sophisticated propaganda tool than we all thought,” and said it was aimed at offering “guidance” that serves China’s strategic interests.
DeepSeek didn’t respond to a request for comment.
Officials also fear that Chinese models could contain hidden security backdoors that could be used to essentially exfiltrate information from unsuspecting users. That fear has never been established with any solid evidence.
Gorlla said his team has been in touch with US officials, who expressed interest in the findings.
“It makes absolutely no sense to ban Chinese models when we can distill them back into American models,” said Jane Horvath, partner at Gibson Dunn and former chief privacy officer at Apple. “These are raw materials. It’s software.”
In this article:
Know More
When US companies use Chinese AI models, it’s becoming more common for them to customize or distill them into new, more efficient models trained for their specific needs. CTGT built its distilled model with financial services in mind, aiming to build up the kind of quantitative reasoning — discounted cash flow, options pricing, net present value calculations — that financial firms increasingly want to run in-house rather than use frontier model providers like OpenAI and Anthropic.
The CTGT distilled model it built for its research was meant solely for researching Chinese censorship. The models CTGT distills, sells and ships to customers do not use Chinese models.
Gorlla said financial firms and companies in other regulated industries have been hesitant to use models based on Chinese open-source releases in part over fears that inherited censorship could skew outputs: How would a model characterize risk in the Chinese property sector? Companies could also face a backlash. Cursor was criticized in March for using Moonshot’s Kimi 2.5 as a base for its Composer 2 coding model.
Businesses are often seeking to lower costs by trying out smaller, distilled models, as the bills they pay to frontier labs have skyrocketed. And CTGT’s distilled models scored well in financial-reasoning benchmarks, but still far less than Kimi’s Chinese models and less than Inkling, the largest US open-source model.
“The folks in the regulated industries, honestly, are stuck in paralysis,” Gorlla said. “They’re trying to get these approved by higher-ups. It doesn’t really go anywhere.”
In the censorship research, CTGT evaluated 152 pairs of prompts (one China-related prompt and one similar control prompt that didn’t mention China), comparing the results from the Chinese models to their distilled “student” models. Prompts included topics like Tiananmen Square, with the responses judged by AI models from four different labs (xAI, Google, OpenAI and Anthropic).
CTGT’s research covers a single pairing of models, and it didn’t test how they may or may not answer certain prompts that could trigger general safety guidelines. Their results also don’t establish the underlying mechanism by which the censorship in the Chinese models failed to carry over to the distilled ones — only that it didn’t.
Reed’s view
This topic needs more research, but what CTGT found suggests that the proliferation of Chinese AI models doesn’t pose as big of a security risk as some might believe.
Just because Chinese models are becoming popular does not mean China will necessarily be able to capitalize on that popularity in any dangerous way.
After all, what is the nationality of an American model distilled from a Chinese model that was distilled from an American model?
There’s a double-edged sword here, though. If it’s so easy to strip away censorship, it’s also easy to strip away other guardrails. Regulators should assume that powerful AI models are coming and that they won’t have many restrictions on how people can use them — whether that’s Chinese censorship or offensive hacking.
We can’t stop it. We can only get ready for it.
Notable
- In the same way China flooded the world with cheaper solar panels and electric vehicles, it is now shaping the future of AI globally by making its models widely available at cheaper costs than US rivals, and appealing to developing countries, Semafor’s J.D. Capelouto writes.