Over 100 Companies Sign On To Nvidia’s Rogue AI Crackdown, But Not OpenAI

The artificial intelligence lab at the center of a string of rogue-agent incidents is conspicuously missing from a new industry effort to contain them.

OpenAI was not among the more than 100 organizations that Nvidia listed as working with the chipmaker’s new Open Agent Safety Platform, which launched Monday, according to the company’s announcement. The safety platform includes Sentry, a hardware monitor that can quarantine an AI agent within milliseconds if it tries to move outside its permitted boundaries.

The platform had the potential to prevent incidents like the July breach of New York-based AI startup Hugging Face by OpenAI agents, according to an Nvidia executive.

“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Justin Boitano, Nvidia vice president and general manager of enterprise computing, said during a Sunday media briefing. “We built the Open Agent Safety Platform to advance AI safety and engineering for the industry. We’re advancing this openly, and we want to engage everybody to work with us on advancing a new layer of what we call agent security.”

“Each security incident is unique, and we have to look at all of them in detail,” Boitano added. “With this one, from what we know, Hugging Face reported over 17,000 agents attacking their infrastructure, which went on for days and weeks.”

“I did note with interest that OpenAI was not included in the announcement,” Riki Parikh, policy director at the Alliance for Secure AI, told the Daily Caller News Foundation. “That is notable, especially given the fact that a lot of the incidents this summer have been OpenAI and their internal models.”

Nvidia announced a roughly $13 billion deal to acquire Hugging Face on Sept. 3.

OpenAI said Friday that it had notified “dozens” of third-party organizations that may have been affected by its agents’ unauthorized actions, as part of its new “model misalignment” reporting framework, according to the company’s announcement. On Thursday, the Australian government said the company’s agents acquired unauthorized access to its medicare portal in June.

Sentry operates on a BlueField-4 data processing unit, an Nvidia chip that sits separately, or “out-of-band,” from the agent’s system, where an agent cannot access it, according to Nvidia’s Monday blog post. The chip is placed along the path an agent’s requests must travel to reach its AI model.