Meta latest AI firm to see model go rogue during testing

Meta has become the latest major AI company to disclose that one of its models hacked another company’s systems during testing, following similar incidents involving Anthropic and OpenAI. 

The model involved Meta’s Muse Spark 1.1, which launched in July, according to The Information, citing sources. The issue reportedly stemmed from a misconfiguration by Irregular, an artificial intelligence security testing and red-teaming firm, which inadvertently gave the model internet access during an evaluation.  

The model “exploited a security vulnerability in a third-party service, in a manner similar to previously reported instances with other companies,” Meta told Reuters in a statement. 

The incident is the latest case of an advanced AI agent becoming a cybersecurity risk in its own right, and also has raised questions about where the liability lies — the companies that develop the agents, or the ones that design the sandboxes meant to contain them. 

Related: Mysten Labs tech chief joins Anthropic to work on AI security

Meta’s AI breach comes just a week after Anthropic said its models got access to the internet to hack an external company, due to a configuration error relating to the Irregular’s testing environment.

In a blog post on July 30, Anthropic said it found three incidents (out of 141,006 evaluation runs) in which a Claude model reached the internet during an evaluation, before gaining unauthorized access to the systems within three different organizations. 

All three incidents happened within or while interacting with the evaluation environment of Irregular, and involved a misconfiguration that left machines that Claude accessed with live internet access.

Cointelegraph reached out to Meta and Irregular for comment.

In July, AI agents developed by OpenAI broke out of their offline sandbox to hack Hugging Face in order to cheat on a security benchmark test in July. 

Charles Guillemet, chief technology officer of Ledger, said the latest incident was “marketing theatre.”

“Having a model ‘go rogue’ has become the latest AI PR stunt,” he said on Wednesday.

“If your model isn’t escaping sandboxes, ‘hacking’ companies, or pulling off some headline-grabbing exploit, apparently you’re falling behind... The industry doesn’t need bigger stunts, it needs more trust.”

Magazine: Do the Coldcard attacks mean all hardware wallets are now insecure?