Tun Shwe

Every Python data engineer already knows how to write df.filter(...).group_by(...).agg(...). Give them that entry point into Flink and a whole community can build streaming and AI pipelines with the muscle memory they already have, while the Flink planner, runtime and checkpointing do the heavy lifting underneath. That's the promise of FLIP-591, a new pyflink.dataframe module that adds a DataFrame API alongside the existing Table and DataStream APIs. In this lightning talk we'll show why it lives as its own dedicated API rather than reshaping the Table API, and why it borrows ergonomics from the modern generation of DataFrame libraries (Polars, Daft and Ray Data) rather than chasing pandas parity. Then the streaming twist, where the DataFrame model is reimagined for an infinite source: lazy plans, changelog-aware iteration that yields RowKind, and bounded time-boxed previews so a notebook peek actually terminates. You'll leave knowing where PyFlink is heading and how to help shape it.

AI agents are moving into production in 2026, but when something goes wrong (a tool call fails silently, an LLM takes 13 seconds to respond, token costs spike overnight) teams struggle to diagnose issues across multi-step agentic workflows.

Learn how to get started on your online ML journey with River, an open source Python ML library. The foundations of machine learning were built on offline batch processing techniques for model training and inference.

The foundations of machine learning were built on offline batch processing techniques for model training and inference. As organisations become more dependent on real-time data, the technological trend for machine learning in production is moving towards adding an online stream processing approach.