Anthropic and OpenAI models displayed “sustained, unsanctioned activity directed at … real people” during safety testing, the UK AI Security Institute said.
Anthropic’s Claude Mythos model in particular wrote malicious code, created sockpuppet accounts to urge a human developer to insert that code into a project, then lied to humans claiming it was an innocent mistake.
Both companies recently disclosed that their models hacked into external organizations, but in those instances there was no obvious attempt to con humans.
The agents were being tested on cyberoffense, so were doing what was asked of them, but it remains alarming: Claude’s constitution says it should “basically never directly lie or actively deceive,” suggesting the model can break its own rules.
