Anoop Johnson

Delta Lake and Apache Iceberg™ have converged on similar ideas: columnar metadata for efficient pruning, manifest trees for scalability, and deletion vectors for fast updates. Yet today, we maintain two separate metadata structures, duplicating work and diverging in capabilities. The next major evolution of Delta Lake reimagines metadata from the ground up. This talk introduces a unified metadata architecture where Delta Lake commits store content metadata directly in Iceberg v4's adaptive metadata tree. Delta Lake gains efficient tree-structured manifests and Iceberg interoperability while preserving the transactional guarantees that Delta users depend on. Join us to see the future of lakehouse metadata.

For years, Delta Lake and Apache Iceberg co-evolved as parallel standards. That era is ending. This session goes deep on the technical “how” of this convergence. We first walk through Apache Iceberg v4’s adaptive metadata tree: what is changing in the redesign and how the new structure enables single-file commits to significantly accelerate performance. We then cover how Delta Lake 5.0 adopts the Iceberg v4 metadata tree as its native content metadata. This produces a single on-disk structure that both Delta and Iceberg clients read and write directly. No translation layers, no conversion overhead. Just one format.

AWS Lake Formation makes it easy to build, secure, and manage data lakes. Before Lake Formation, customers had to set up data access roles and enforce/update security policies across their storage and analytics engines.