Frank Munz

This introductory workshop caters to data engineers seeking hands-on experience and data architects looking to deepen their knowledge. The workshop provides a solid understanding of the following fundamental data engineering concepts: - Introduction to Lakeflow and the Data Intelligence Platform - Getting started with Spark Declarative Pipelines, Streaming Tables, and Materialized Views - Working with a realistic data streaming use case - Mastering Lakeflow Jobs with advanced control flow and triggers - Creating pipelines with Genie Code and Claude Code - Real-time Mode (RTM) - Understanding data governance and lineage with Unity Catalog - Benefits of Serverless Compute. We'll equip you with your own lab environment in this workshop.

This Data + AI Summit session walks you through a streaming application that analyzes live NASA space data using Lakeflow and Agent Bricks. Lakeflow is the foundation layer of every AI application. I explain how Lakeflow Spark Declarative Pipelines (SDP) ingests from one of the very few public Kafka topics that exist and how it works in symbiosis with Agent Bricks to turn real-time NASA circulars about cosmic events into an AI-powered knowledge base with Databricks Apps UI. The AI stack covers AI Gateway, guardrails, Vector Search, agent evaluation, and MLflow tracing, all from a data engineer's perspective. The second part focuses on Time to Value. I used several Databricks AI tools: Genie Code, Claude Code, the AI Dev Kit, and others. Each capable of taking you from zero to a working application in days rather than months. I share the trade-offs and best practices learned. Grab the full application from GitHub afterward — and play with data from exploding supernovas.

This session builds on the foundational OSS technologies of the modern Lakehouse—Apache Kafka, Spark, Unity Catalog and MLFlow—and shows how everyone can analyze supernova data coming from NASA's satellites and analyze data streams with natural language and plot their own map cosmic events.

This introductory workshop caters to data engineers seeking hands-on experience and data architects looking to deepen their knowledge. The workshop is structured to provide a solid understanding of the following data engineering and streaming concepts: - Introduction to Lakeflow and the Data Intelligence Platform - Getting started with Lakeflow Declarative Pipelines for declarative data pipelines in SQL using Streaming Tables and Materialized Views - Mastering Databricks Workflows with advanced control flow and triggers - Understanding serverless compute - Data governance and lineage with Unity Catalog - Generative AI for Data Engineers: Genie and Databricks Assistant We believe you can only become an expert if you work on real problems and gain hands-on experience. Therefore, we will equip you with your own lab environment in this workshop and guide you through practical exercises like using GitHub, ingesting data from various sources, creating batch and streaming data pipelines, and more.

This session is repeated. This introductory workshop caters to data engineers seeking hands-on experience and data architects looking to deepen their knowledge. The workshop is structured to provide a solid understanding of the following data engineering and streaming concepts: - Introduction to Lakeflow and the Data Intelligence Platform - Getting started with Lakeflow Declarative Pipelines for declarative data pipelines in SQL using Streaming Tables and Materialized Views - Mastering Databricks Workflows with advanced control flow and triggers - Understanding serverless compute - Data governance and lineage with Unity Catalog - Generative AI for Data Engineers: Genie and Databricks Assistant We believe you can only become an expert if you work on real problems and gain hands-on experience. Therefore, we will equip you with your own lab environment in this workshop and guide you through practical exercises like using GitHub, ingesting data from various sources, creating batch and streaming data pipelines, and more.