Anshul Gupta

Modern AI applications don’t just need accurate models, they need to serve them at massive scale, with low latency and strict reliability guarantees. In this session, we’ll go under the hood of Databricks Model Serving to show how we support high QPS (300K+ QPS), production-grade workloads on a unified Lakehouse platform. We’ll walk through the serving architecture, autoscaling strategies, and optimizations across GPU/CPU utilization, request routing, and caching that enable sustained high throughput without sacrificing latency or cost. You’ll see real patterns from customer deployments, learn how to design your endpoints for bursty and always-on traffic, and leave with practical guidance for running mission-critical ML and LLM workloads on Databricks Model Serving.

Shipping AI products is easy. Trusting and operationalizing them at production scale is the hard part. Operational observability is now having its agentic moment. Join us to see how Databricks is reinventing operational observability for the agentic era — a unified, production-scale telemetry layer powered by Genie Code, our AI assistant, spanning every AI offering on the platform: Model Serving, AI Search, AI Gateway, Agent Bricks and Apps. Genie Code doesn't just alert you that something's wrong — it delivers instant root-cause analysis, pinpoints fixes for new error patterns and runs agentic operations on demand. Walk away knowing how to keep your application running smoothly in production — and how to get to the bottom of any issue in seconds when it doesn't.