Robert Pack

There's a quiet crisis going on in open source right now — whenever an open table format wants to build a new feature or integration, there's an exponentially more amount of work to do to incorporate it. A project that's increasingly harder to contribute to is the death of innovation. Kernel sets out to solve this problem by rewriting the source of truth. When other projects adopt Kernel, they no longer need to do any additional development for new features — they just appear. Join us for a deep dive into how we turned this project on its head, to solve not just these problems, but future problems we anticipate in open source. What started out as a contribution to Delta, could change the way that open table formats fundamentally integrate everywhere. Join Robert Pack, open-source contributor for Delta, Iceberg, Arrow and all round Rust enthusiast, and Holly Smith, tech communicator, who bullied Robert into this talk.

You can build a serious data-and-AI platform almost entirely from open source today. We'll grow one alongside a business — open storage and Delta, **Unity Catalog OSS**, multi-engine access on the same managed tables via the new **UC Delta APIs**, MLflow, lineage. The Open Lakehouse is real — and it already governs data well: credential vending, server-side planning, trusted compute. What those patterns share is an assumption — every request comes from a human you can hold accountable — and agents open a gap in it, reading what they're allowed to then acting in ways no one authorized, while the catalog that governed the read never sees the act. We'll show today's open governance patterns, where agents and cross-org sharing strain them, and why closing the gap — including with the new **OpenSharing** protocol — means extending governance to the platform, in the open. Context is the new data.

Join us as we introduce Delta-Kernel-RS, a new Rust implementation of the Delta Lake protocol designed for unparalleled interoperability across query engines. In this session, we will explore how maintaining a native implementation of the Delta specification — with native C and C++ FFI support — can deliver consistent benefits across diverse data processing systems, eliminating the need for repetitive, engine-specific reimplementations. We will dive deep into a real-world case study where a query engine harnessed Delta-Kernel-RS to unlock significant data skipping improvements — enhancements achieved “for free” by leveraging the kernel. Attendees will gain insights into the architectural decisions, interoperability strategies and the practical impact of this innovation on performance and development efficiency in modern data ecosystems.

Five years ago, the delta-rs project embarked on a journey to bring Delta Lake's robust capabilities to the Rust & Python ecosystem. In this talk, we'll delve into the triumphs, tribulations and lessons learned along the way. We'll explore how delta-rs has matured alongside the thriving Rust data ecosystem, adapting to its evolving landscape and overcoming the challenges of maintaining a complex data project. Join us as we share insights into the project's evolution, the symbiotic relationship between delta-rs and the Rust community, and the current hurdles and future directions that lie ahead. Audio for this session is delivered in the conference mobile app, you must bring your own headphones to listen.

Join us for an in-depth Ask Me Anything (AMA) on how Rust is revolutionizing Lakehouse formats like Delta Lake and Apache Iceberg through projects like delta-rs and iceberg-rs! Discover how Rust’s memory safety, zero-cost abstractions and fearless concurrency unlock faster development and higher-performance data operations. Whether you’re a data engineer, Rustacean or Lakehouse enthusiast, bring your questions on how Rust is shaping the future of open table formats!