Ananya Roy

As LLM applications move into production, redundant calls quietly drive up latency, cost, and complexity. Traditional caching falls short — exact string matches miss the majority of real-world queries where users ask the same question in different ways.In this session, we introduce semantic caching: a smarter approach that retrieves responses based on intent, not syntax. You'll learn how to build a production-ready semantic cache gateway using Lakebase and MLflow to capture paraphrased and context-aware queries. We'll walk through why exact-match caching breaks down at scale, how to design a low-latency caching layer with a FastAPI gateway on Databricks Apps, and how to incorporate conversational context for multi-turn accuracy — plus the observability and governance patterns that make it production-safe.Walk away with a deployable architecture and proven patterns to cut LLM costs by up to 80% while improving response speed and reliability.

Most AI agents break when they leave the lab. This session shows you how to build ones that don't — using Databricks as your security-first production backbone.You'll see battle-tested architectural patterns for three real-world scenarios: internal productivity agents with least-privilege access, agents that securely connect to external tools and MCP servers, and governed B2C agents serving non-Databricks users through external web apps.At the core: Unity Catalog as your governance engine. We go deep on passing OAuth identity claims into UC functions for row-level data retrieval, applying identity-based filters within Vector Search, and enforcing tool-level restrictions that travel with your agent across every access pattern.You'll leave with: a reusable architectural blueprint for production agent deployment, concrete techniques for identity-aware data access, and a governance model that scales without becoming a bottleneck.Start shipping agents you can trust with Databricks.

Generic LLM metrics are useless until it meets your business needs.In this session we will dive deep into creating bespoke custom state-of-the-art AI metrics that matters to you. Discuss best practices on LLM evaluation strategies, when to use LLM judge vs. statistical metrics and many more. Through a live demo using Mosaic AI Framework, we will showcase: - How you can build your own custom AI metric tailored to your needs for your GenAI application - Implement autonomous AI evaluation suite for complex, multi-agent systems - Generate ground truth data at scale and production monitoring strategies - Drawing from extensive experience on working with customers on real-world use cases, we will share actionable insights on building a robust AI evaluation framework By the end of this session, you'll be equipped to create AI solutions that are not only powerful but also relevant to your organizations needs. Join us to transform your AI strategy and make a tangible impact on your business!