Drew Gallardo

Today, committing data to an Iceberg table requires clients to construct a full set of metadata, even when the underlying change is as simple as appending a small set of data files. While this process may work well for fully fledged out engines like Spark, it creates a high barrier for light weight services that can produce files like Parquet but don’t implement Iceberg’s Snapshot infrastructure.

PyIceberg has come a long way in the past year, crossing 500k daily downloads on PyPI, with many teams now running it in production. In this talk, we’ll highlight some of the new community-driven features that make working with Apache Iceberg tables in Python both easy and fun.