Matthew McCoy

This hands-on course introduces learners to evaluating, governing, and securing agentic AI systems on Databricks. You'll begin by exploring the motivation for evaluation and governance frameworks and how they connect to the Databricks Data Intelligence Platform. Next, you'll learn how to apply MLflow evaluation metrics and a variety of evaluation techniques, including online evaluation, synthetic evaluation, and building evaluation datasets from MLflow traces. The course then examines how Unity Catalog governance extends to agents' functions, models, and tools, ensuring proper access control, auditability, and compliance. Finally, you'll learn Databricks AI Security Framework (DASF) and how to secure agents with Mosaic AI Gateway, applying guardrails, rate limits, and policy filters to enforce safe and reliable usage. Note: Hands-on training courses will be updated to reflect the newest product and feature announcements from Data + AI Summit in June 2026. Prerequisites - Familiarity with the Databricks Data Intelligence Platform, including Unity Catalog for governance and model management - Intermediate Python programming experience - Familiarity with Generative AI fundamentals,…

This hands-on course teaches you how to build retrieval-augmented generation (RAG) agents on Databricks using Agent Bricks’ tooling—including Vector Search, AI Playground, and the Foundation Model APIs. First, you'll begin with prompt engineering and the fundamentals of retrieval and augmented generation. Next, you'll learn how to ingest, process, and prepare documents for use in a retrieval pipeline. Then you'll explore embedding vectors, vector database design, and how to use Vector Search in a Databricks environment. Finally, you will build a retrieval agent end-to-end and trace its behavior using MLflow Tracing for observability and debugging. Note: Hands-on training courses will be updated to reflect the newest product and feature announcements from Data + AI Summit in June 2026. Prerequisites - Familiarity with the Databricks Data Intelligence Platform, including Unity Catalog - Intermediate Python programming experience - Basic SQL proficiency for querying and creating functions - Familiarity with Generative AI fundamentals, including large language models (LLMs) and prompt engineering - Basic knowledge of MLflow