Wenwen Xie

Discover how Databricks transformed a basic Gradio app into GTM Hub 2.0, an enterprise-grade platform serving 10,000+ global users (Sales, Marketing, XDR) with personalized insights. Core Architecture & Achievements: - Databricks Apps (Enterprise Scale): Built a React/FastAPI application serving 10k+ concurrent users with <3s load times. Features include RBAC, multi-persona views, and seamless cross-origin embedding. - AI/BI Dashboards: Embedded dynamic dashboards with global filters, bookmarkable states, and usage tracking for optimized analytics. - Genie (Conversational AI): Integrated multi-agent support. A supervisor agent (via MLFlow) routes natural language queries across specialized spaces, eliminating the need for SQL. - Lakebase (State Management): Leveraged PostgreSQL on Databricks to handle user preferences, session history, and caching for real-time personalization. The Result: A unified, modular analytics experience with robust telemetry and a future-proof architecture.

If your team is building agents, evaluation is the difference between a demo that works and a product you can actually rely on. But most teams don't know where to start — which metrics matter, how to build a benchmark, how to automate tests, or how to catch regressions before they reach users.In this how-to session, we walk through agent evaluation end-to-end: defining your first benchmark, choosing the right judges and metrics for your use case, automating evaluation across prompts and workflows, and building a CI/CD loop that catches regressions before they hit production. We'll share practical patterns from real production teams and show how MLflow ties it all together.You'll leave with a concrete playbook for evaluating agents from first test to continuous quality at scale — and the confidence to ship agents that keep getting better with every iteration.

In enterprise AI, Evaluation-Driven Development (EDD) ensures reliable, efficient systems by embedding continuous assessment and improvement into the AI development lifecycle. High-quality evaluation datasets are created using techniques like document analysis, synthetic data generation via Mosaic AI’s synthetic data generation API, SME validation, and relevance filtering, reducing manual effort and accelerating workflows. EDD focuses on metrics such as context relevance, groundedness, and response accuracy to identify and address issues like retrieval errors or model limitations. Custom LLM judges, tailored to domain-specific needs like PII detection or tone assessment, enhance evaluations. By leveraging tools like Mosaic AI Agent Framework and Agent Evaluation, MLflow, EDD automates data tracking, streamlines workflows, and quantifies improvements, transforming AI development for delivering scalable, high-performing systems that drive measurable organizational value.