Dado Ruvic/File Photo/ReutersAli Ghodsi, CEO of Databricks, believes AI agents could transform business. There’s just one problem: For a lot of Databricks’ customers, AI agents are failing, and Ghodsi says he knows why. Companies are, essentially, overcomplicating things by trying methods like “fine tuning” AI models based on proprietary data, and then using techniques to corral AI models into a narrow set of responses to eliminate erroneous answers. It takes trillions of tokens, or bits of data, to train an AI model. But companies only have a tiny fraction of that amount of data. In essence, fine tuning only goes so far. On top of that, techniques change every day. Once a company implements a fine-tuned AI model, it is outdated the very next day. Databricks’ new philosophy is to keep it simple. Instead of customize, companies should try out a lot of different AI models and see which one is best for specific goals. With AI models improving in capability every couple of weeks, what doesn’t work today might work very soon. Databricks announced a handful of new products at its Data + AI summit this week that offer this philosophy as a service. “We have a whole cookbook that we’re throwing in there. We’ll see which one does best. And there’ll be new techniques, you know, and new models, and so on,” Ghodsi told me earlier this week. “Whatever you come up with that you think is really smart, the next day, someone [will] say, ‘You need to see this thing on Twitter yesterday.’” This is, of course, a product announcement. At the same time, it says something about the blistering speed of the AI revolution. Whatever you build today, you might have to throw out in a few months. Companies are going to spend a lot of money on technology that will end up on the cutting-room floor. |