US President Donald Trump floated the possibility of further controls on AI development in the wake of an OpenAI model going rogue and industry leaders joining calls for a pause. More than 1,200 AI company employees — including Anthropic’s CEO and senior Google DeepMind and OpenAI executives — signed a “Pacing the Frontier” petition calling for “technical and governance tools” to “buy time to address emerging risks.”
Those may be severe: 272 experts surveyed by MIT thought 18 of 24 potential risk categories, including AI models developing dangerous capabilities or providing them to users, had a more than 10% chance of causing “catastrophic” outcomes, meaning 1 million deaths, $100 billion in losses, or “civilization-scale” harms, before 2030.