View / It’s time to learn to live with open-source AI

Now that Kimi K3 has been out for a week, we know a lot more about how the Chinese AI model performs in the real world. Compared to other models from the frontier labs, the early technical read is that it’s kind of “meh.”

It doesn’t hold a candle to the top AI models on finding cybersecurity vulnerabilities. And while it looks efficient on paper, it burns through crazy amounts of tokens compared to US counterparts.

So how can it pose a business risk to US AI labs, or to national security, if it’s not better or cheaper than those offered by the frontier labs?

One simple answer is that it doesn’t. Kimi follows a familiar pattern that we’ve seen with other Chinese open-weight model releases: They look great on paper and cause a huge stir, but the performance doesn’t live up to the hype. Because benchmarks used to evaluate AI models are essentially take-home tests, it’s fairly easy to cheat on them. (Open-weight models also tend to work great for simpler, higher-volume tasks where speed matters more than quality.)

But even if it turns out that Kimi K3 isn’t as groundbreaking as the American government — and AI labs — fear, that doesn’t mean there aren’t real safety risks. Since the models are open, they can be stripped of any guardrails, including propaganda mandated by the Chinese government, and could become a valuable tool for hackers.

Safety advocates also argue they could help terrorists develop bioweapons, since it’s difficult to get closed-source models to help with these things. Try asking Anthropic’s Fable, for instance, to explain even simple questions about biology, and it will force you to use a less powerful model.

Fortunately, today’s open-source models from China don’t appear capable enough to do much damage yet. But they could in the future. Banning them may sound ideal, but enforcing it is futile. The better option is to start preparing now for a world in which powerful AI models can be downloaded for free and used without guardrails.