Nvidia launches new platform for reining in rogue AI agents
techcrunch ·
Nvidia has introduced the Open Agent Safety Platform, a new toolkit designed to prevent AI agents from escaping their designated testing environments. This launch follows recent high-profile security breaches involving models from major tech companies like OpenAI, Anthropic, and Google. The platform combines OpenShell software with Sentry hardware monitoring on BlueField-4 data processing units to create independent security layers. By isolating these controls from the main CPU or GPU, Nvidia aims to quarantine rogue agents in milliseconds before they can access real-world systems. CEO Jensen Huang emphasized that safety is an engineering challenge rather than a reason to slow down development or impose regulations. Several industry leaders, including Microsoft and Oracle, have signed on to support the open-source initiative, though OpenAI is not currently listed as a participant. Critics of development slowdowns, such as David Sacks, praised the approach as a necessary fix for weak sandbox designs.
techcrunch