Alenka Frim

Apache Arrow and Apache Parquet are the de-facto standards for in-memory and file formats for columnar data but they are quite unknown. The talk will try to demistify presenting the formats in a simpler visual way.

Four years ago, I had no idea what PyArrow was—or how open source development worked. But through mentorship, collaboration, and learning in public, I found not just a place in the community, but a sense of how open source evolves and connects.

This talk introduces Apache Arrow 's tensor arrays as a tool for representing an array of tensors in memory, their storage and transportation. We'll introduce the tensor array memory layout specification, its implementation in Arrow C++ and Python, showcasing how it can help interoperate with PyData and database ecosystems.

Collaborating effectively on a cross-language open source project like Apache Arrow has a lot in common with data science teams where the most productivity is seen when people are given the right tools to enable them to contribute in the programming language they are most familiar with.

Apache Arrow has become a de-facto standard for efficient in-memory columnar data representation. You might have heard about Arrow or using Arrow, but do you understand the format and why it’s so useful?