Will Manning

This talk will cover the Vortex, a Linux Foundation project building the fastest open-source file format on both CPUs and GPUs. We'll briefly cover Vortex's performance, and then break down how we built the format to support efficient late materialization on both GPUs & CPUs.

To future-proof Apache Iceberg for emerging analytics needs, we presents a File API abstraction (now in late-stage community review) that allows file-format flexibility and seamless integration of next-generation storage formats. We demonstrate its power with Vortex, a performant and extensible columnar format built for both analytics & AI/ML use cases.