Reed’s view
It isn’t just big companies fretting about high and unexpected bills for AI tokens. It is also … me. On Sunday morning, I woke up jet-lagged after two weeks in Europe to find that OpenAI was repeatedly charging me $5, more than 20 times a day in some cases. I had no idea why.
I’ve been using OpenAI’s Codex desktop app for months, which has helped me personally and professionally and even led to the launch of a new product called Semafor Intelligence. Before I left, my AI spend was pretty low. I’m not a tokenmaxxer or a pretend software developer, and I had never bothered to set limits on how often Codex could “top up” with $5 credit recharges.
As my blood pressure rose, I turned to Codex, where the problem started about a month ago. A software glitch had erased all of my Codex chats.
After deleting and reinstalling it, I asked Codex to revive a particularly ambitious project, which somehow sent my agent into a loop where it kept reviewing a bunch of data over and over again, racking up tokens. This was happening on a Mac Mini that I access remotely. And, because of consistently spotty data connection overseas, I wasn’t checking it.
Codex explained that the agent had taken the prompt for my ambitious project “too literally.” (Codex had written it, mind you.) In the end, I was billed just shy of $500. It could have been worse.
I asked Codex to help again — this time to contact OpenAI’s customer service department and explain what happened, hoping they might refund me. Soon, I was a messenger, shuttling responses back and forth between Codex and OpenAI’s customer support AI chatbot.
In the end, there was no refund, and I never spoke to a real person at OpenAI until I contacted their PR department. I am the only human in this story.
My bill was entirely avoidable. I made lots of mistakes, including putting too much faith in an unproven technology and giving it too much control and access to my wallet. I’m as much a software developer as a celebrity sitting courtside at a Knicks game is a professional basketball player. Today, this technology is the software equivalent of a fantasy camp.
Whether you’re a big company or a lowly journalist, the price of playing with this technology now is a lot of token waste and overspending. But there’s no other way to get a glimpse into the future, and this is the trade-off that everyone is grappling with. Just do yourself a favor: Put a limit on those auto-credit top-ups.
Notable
- It’s not just journalists; token costs are exceeding some employees’ salaries at JPMorgan, the bank’s chief data and analytics officer at its Payments division told Semafor last month.