Nishith Sinha

The pace of AI development has reset what security teams need to deliver. At Databricks, we've built an AI Security team that operates at the frontier of the technology it defends. This session is an inside look at how. We'll share the human-in-the-loop agents we built to assess security issues across our AI products and services, and why an early, disciplined bet on agent benchmarking got us to 95.8% accuracy on agent-generated findings, and how we use expert reviewers validating every result so nothing slips through. Then we'll open the hood on how we secure the coding agents our own engineers use every day: managed client-side settings, agent sandboxes, an AI Gateway and MCP Gateway enforcing policy server-side where admin access can't reach, and continuous scanning of every Databricks App, plugin, and skill.Real architecture. Real numbers. Real failure modes. A practical look at what a modern AI security function looks like in production.

Agentic AI is moving from POCs to mission-critical workloads- and the attack surface is expanding fast. As LLM-driven agents reason, plan, and act across enterprise data and tools, a new class of risk emerges: the "Lethal Trifecta," where sensitive data access, untrusted inputs, and external actions converge to make prompt injection attacks exploitable end-to-end. Karol Piekarski (Experian) and Nishith Sinha (Databricks) present the Databricks Agentic AI Security Framework (DASF v3), drawing on both Databricks' own AI infrastructure which handles millions of daily queries, and patterns from Fortune 100 deployments. The session walks through how the Databricks platform breaks the Lethal Trifecta at each layer: constraining what agents can access, hardening how they process inputs, and limiting what a compromised agent can actually do. Finally, see how the Databricks Security team built, open-sourced, and actively uses BlackIce, a containerized AI red teaming toolkit, to shift security from a point-in-time exercise to continuous, automated validation. Leave with a practical framework for moving fast on AI without compromising governance or security.