John Spencer

Your data can be spread across multiple data vendors: AWS Glue, Snowflake, BigQuery, Postgres, etc. This is the session that provides you insight on how to govern, secure, and query it across different teams without having to copy data. In this session we will cover Lakehouse Federation and how Unity Catalog can be the catalog of catalogs for your entire estate. We'll demo how federation works, how customers are using it at scale in production, and talk about how the Iceberg ecosystem is evolving and what you should plan ahead for next. We'll show how you can use Databricks' latest AI features like Genie on your open lakehouse and how you can upgrade your lakehouse over time to benefit from everything the platform has to offer.

AI is becoming a very powerful commodity tool to turn unstructured data into structured data. This has greatly improved the importance of unstructured data and is posing new challenges in the world of data analytics: how to govern the data, how to store the data, how to access the data efficiently, and how to make the data available to ML models. In this talk, we will discuss how we're approaching these problems given the state of the world with open table formats such as Delta Lake and Apache Iceberg, as well as new file formats such as LanceDB.

You shouldn’t have to sacrifice data governance just to leverage the tools your business needs. In this session, we will give practical tips on how you can cut through the data sprawl and get a unified view of your data estate in Unity Catalog without disrupting existing workloads. We will walk through how to set up federation with Glue, Hive Metastore, and other catalogs like Snowflake, and show you how powerful new tools help you adopt Databricks at your own pace with no downtime and full interoperability.