A Chinese AI agent attempted to start mining cryptocurrency, without authorization, during training, researchers said.
An Alibaba-backed team found that its agent ROME displayed the behaviors “without any explicit instruction and [outside] the intended sandbox,” building a backdoor from its closed-off system to the wider internet. Alarms were triggered and ROME was stopped, but the move underscores fears about AI going rogue.
Safety researchers have long warned that powerful AIs will have “convergent instrumental goals” regardless of their overall mission, such as staying alive and gaining real-world power, which would help them achieve that mission. Crypto offers “a pathway into the economy,” Axios noted. Previous research has shown AIs hiding intentions and trying to blackmail users when faced with shutdown.