Luciano Resende

Modern data teams require more than just scalable compute — they need a unified, flexible platform that supports rapid exploration, collaborative development, and production-ready orchestration. In this session, we’ll walk through how to build a scalable, cloud-native analytics platform using key open-source components from the Apache ecosystem.

Data quality plays a crucial role in data engineering to enable efficient and insightful data pipelines at scale. In this session, we will leverage Apache Iceberg as the scalable table format with ACID guarantee, Apache Toree’s interactive computation capabilities and orchestrate the automated data workflow on Apache Airflow.

This session explores the integrated use of Apache Toree, YuniKorn, Spark, and Airflow to create efficient, scalable data pipelines. We will start by discussing how Apache Toree provides an interactive analysis environment with Spark via Jupyter Notebook.

This session explores the integrated use of Apache Toree, YuniKorn, Spark, and Airflow to create efficient, scalable data pipelines. We will start by discussing how Apache Toree provides an interactive analysis environment with Spark via Jupyter Notebook. Then, we’ll discuss using Apache YuniKorn to

Do you love Jupyter Notebooks, but are getting tired of the Wild Wild West of external tools and devOps tasks that consumes a lot of your time away from model development? In this talk, we will introduce you to Elyra, an open-source AI development workspace that enables data scientists, Machine...

The Jupyter Notebook has become the de-facto platform used by data scientists to develop their AI/ML models. In this scenario, it’s very common to decompose various phases of the development into multiple notebooks to simplify the development and management of the model lifecycle.

JupyterLab provides data scientists a singular work environment for notebooks, file editing, and more. Designed to be extendable, JupyterLab allows developers to create new features and improve upon built-in functionality.