Jenny Sun

We're often told that Vector Databases are the "memory" for AI Agents. But vector search only gives you a probabilistic similarity score. In enterprise workflows, "probably" isn't good enough. Agents need deterministic, transactional state—and they need the ability to compound their knowledge over time. In this session, we'll move beyond stateless "goldfish" chatbots and build a truly self-improving agent system using Databricks Apps, Databricks Jobs, and Lakebase Autoscaling. We'll demonstrate how to build a production-ready "Stateful Worker" agent, featuring asynchronous batch "dreaming" distillation jobs that allow your agents to distill insights and get smarter overnight. To ground these concepts, we'll walk through the architecture from three critical perspectives: - The Agent User Experience: How can we ensure an agent knows a user's role, preferences, and past context? We will demo the stateless stateful agents on Databricks Apps and see how users can benefit from user and agent scoped memory. - The Database Admin Experience: How do we audit what an agent is learning? We'll explore Lakebase's intuitive UI/SQL, understand the Agent's thinking with observability tools, and…