Gábor Kaszab

Machine learning and feature engineering workloads increasingly rely on very wide tables with thousands of columns, exposing fundamental limits in current columnar layouts. These limits increase I/O and metadata complexity while slowing down query planning.

As table metadata grows, loading tables becomes an increasingly expensive operation, often affecting query latency and overall engine scalability. This talk explores how query engines can mitigate this cost by partially loading table metadata from a REST catalog, and caching metadata on the client side.

Apache Impala is a horizontally scalable database engine renowned for its emphasis on query efficiency. Over recent years, the Impala team has dedicated substantial effort to support Iceberg tables.