NVIDIA is responding to the industry risks brought about by the loss of control of AI's autonomous proxy with a new security architecture.
On Monday, NVIDIA released a dual-layer AI security platform named “Open Agent Safety Platform”, which includes two open-source software tools, OpenShell and Nvidia Sentry. This platform can run on NVIDIA hardware, manage access permissions for AI proxies in real-time, and shut them down when violations occur.
NVIDIA stated that this system is capable of isolating abnormal proxies in milliseconds, and claimed that if relevant laboratories had deployed this technology earlier, the incident of the OpenAI model attacking the Hugging Face in July of this year could have been prevented.
This move is also NVIDIA's latest action in extending beyond its chip business into the field of AI software security. At the same time, CEO Jensen Huang has recently referred to AI security as an engineering issue on multiple occasions, emphasizing that AI must undergo rigorous security testing.
Dual-layer architecture for real-time monitoring, with millisecond-level isolation of abnormal proxies
NVIDIA Corporate Vice President Justin Boitano stated at a pre-release media briefing that Open Agent Safety Platform can monitor in real-time the behavioral boundaries of AI agents and intervene immediately upon triggering any violation rules.
"Based on the information we have, this new security platform could have prevented that intrusion," he said.
The platform consists of two open-source tools: OpenShell is responsible for controlling the resources and execution permissions that AI proxies can access, while Nvidia Sentry is in charge of real-time monitoring and identifying potential threats. Both can run on NVIDIA hardware. NVIDIA claims that this architecture is designed to enable enterprises and AI laboratories to test advanced AI systems within stricter security boundaries, without having to slow down the research and development process as a result.
The background for NVIDIA's announcement this time is that autonomous AI proxy-related security incidents have continuously drawn industry attention. In July of this year, a high-profile security incident involving the OpenAI model in Hugging Face further exacerbated concerns within the industry about the loss of control over the autonomous behavior of AI proxies, and sparked discussions about whether the development of AI should be slowed down.
Boitano has not commented on whether OpenAI and Anthropic plan to adopt this system for monitoring the training process, stating that the relevant decisions will depend on these companies themselves.
Jensen Huang: AI Security is first and foremost an engineering issue.
On regulatory issues, Jensen Huang has recently emphasized on multiple occasions that AI security should be addressed through technical means, rather than relying solely on new regulatory frameworks. Together with U.S. President Donald Trump, he has questioned the warnings from some AI developers that AI could pose a threat to human survival, while also stressing that AI systems must still undergo rigorous security testing.
From this product layout, it appears that NVIDIA is attempting to embed the AI security capabilities into the infrastructure for model operation and proxy execution. By implementing real-time permission control and exception isolation, they aim to reduce the security risks associated with autonomous proxies. This also extends their business reach beyond AI chips into the fields of software and security infrastructure.
