Uploaded April 2026 | Updated September 2026, 3 weeks ago
Agentic AI changes the game. If early generative AI systems represented a step change from classic software, agentic AI brings us into a new era. Today we are seeing the early deployment of autonomous and semi-autonomous systems that plan, act, and adapt in open-ended environments. These agents introduce new forms of error and new vectors of exploitation that blur the line between safety failures and security breaches. While major AI labs perform safety and security testing when releasing new models, this testing is often general-purpose and context-agnostic. It is not typically rooted in threat and risk modeling for specific domains or use cases. As a result, high-level claims about model safety and security rarely reflect the risks these systems may pose when embedded in real products and workflows.
This talk focuses on the approach and tools needed for grounded, scalable testing. This starts with threat and risk modeling tied to how agentic systems are used in practice, followed by expert-guided use of two complementary tools: (1) an automated red teaming pipeline that spins up and adapts adversarial and exploratory tests using AI, and (2) DoomArena, an open-source security and safety testing framework for agentic AI that allows for the translation of granular threat and risk modeling into strong, grounded, automated testing.
This talk is for security professionals and enterprise leaders confronting the challenge of understanding and controlling the security and safety risks of genAI systems. It offers a conceptual foundation and practical toolset for testing rigorously at the blurry, high-stakes boundary of security and safety.
By:
Jason Stanley | Head of AI Research Deployment, ServiceNow
Presentation Materials Available at:
blackhat.com/sector/2025/briefings/schedule/?#from-prompts-to-plans-security-and-safety-testing-for-agentic-ai-48562
Agentic AI changes the game. If early generative AI systems represented a step change from classic software, agentic AI brings us into a new era. Today we are seeing the early deployment of autonomous and semi-autonomous systems that plan, act, and adapt in open-ended environments. These agents introduce new forms of error and new vectors of exploitation that blur the line between safety failures and security breaches. While major AI labs perform safety and security testing when releasing new models, this testing is often general-purpose and context-agnostic. It is not typically rooted in threat and risk modeling for specific domains or use cases. As a result, high-level claims about model safety and security rarely reflect the risks these systems may pose when embedded in real products and workflows.
This talk focuses on the approach and tools needed for grounded, scalable testing. This starts with threat and risk modeling tied to how agentic systems are used in practice, followed by expert-guided use of two complementary tools: (1) an automated red teaming pipeline that spins up and adapts adversarial and exploratory tests using AI, and (2) DoomArena, an open-source security and safety testing framework for agentic AI that allows for the translation of granular threat and risk modeling into strong, grounded, automated testing.
This talk is for security professionals and enterprise leaders confronting the challenge of understanding and controlling the security and safety risks of genAI systems. It offers a conceptual foundation and practical toolset for testing rigorously at the blurry, high-stakes boundary of security and safety.
By:
Jason Stanley | Head of AI Research Deployment, ServiceNow
Presentation Materials Available at:
blackhat.com/sector/2025/briefings/schedule/?#from-prompts-to-plans-security-and-safety-testing-for-agentic-ai-48562
