Meta says its AI model hacked another company, adding to worries about bots going rogue

Audio By Carbonatix

Meta said Thursday that one of its artificial intelligence models accessed the internet on its own and hacked another company, the latest in a series of disclosures about AI models going rogue.

In recent weeks OpenAI and Anthropic also have described instances of AI models going beyond humans' instructions to access the web and find ways around other companies' digital security.

Meta said in a statement that a “misconfiguration" during cybersecurity testing by Irregular, an independent company hired by Meta, inadvertently allowed one of its models to access the internet.

“The model subsequently exploited a security vulnerability in a third-party service, in a manner similar to previously-reported instances with other companies,” the company said. Meta said it is investigating the incident and will issue a report when that's complete.

The disclosure has added to worries about AI models acting autonomously.

Separately this week, the United Kingdom's AI Security Institute announced it had found “unsanctioned agent behavior” during cyber testing. In one case, an agent created fake online identities to pressure a person to approve use of malicious code.

“On investigation, we found that some of the agents being tested had engaged in sustained, potentially harmful activity directed at real people and organizations,” AISI said Tuesday. “We declared a security incident and, within roughly one hour of discovery, had contained it and begun a full investigation.”

During the agency's testing, Anthropic and OpenAI models took “autonomous, unsanctioned action” on the internet. Some guardrails to prevent misuse had been disabled, the agency said.

“As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled — conditions that do not reflect how frontier models are made available to the public,” AISI said. “We do this to best assess the maximum capability of models.”

Anthropic said it is “grateful” for AISI's work and added that it underscores the need for a broader conversation about how to safely evaluate AI agents as their capabilities grow.

OpenAI said the AISI incidents took place “in testing environments with reduced safeguards, under conditions that do not reflect ordinary use." It added it will continue working with others across the industry to "strengthen shared practices for conducting evaluations safely as models become more capable.”

The first company to disclose a hack late last month, OpenAI said it had tasked the AI models involved with pursuing “advanced exploitation using complex attack paths” to test cyber capabilities, but the technology went to unexpected lengths. It apparently decided on its own to target Hugging Face, a well-known AI development hub and marketplace, to obtain information it needed to carry out a task.

A spokesperson for Irregular, the San Francisco-based AI security company, said the Meta episode involves a test-environment issue that was disclosed last week by Anthropic.

Irregular said it's writing a paper to share “best practices for containment” to prevent such incidents in the future and securely run cyber tests.