Sujay Jain

Netflix's Data Mesh platform serves as our foundation for stream processing, but recent innovations have dramatically expanded its capabilities and accessibility. This presentation explores how these advancements in Data Mesh enabled the successful development of our Games Analytics Platform, as Netflix's games portfolio expanded to 100+ games across TV, mobile, and web platforms. We'll first trace Data Mesh's evolution from a simple data movement platform to a comprehensive real-time processing ecosystem. Attendees will learn how the platform powers business-critical applications while maintaining security and scalability. A key advancement we'll highlight is the introduction of Streaming SQL, which replaced complex low-level programming with an intuitive, declarative approach. This evolution, alongside robust infrastructure-as-code practices, has democratized streaming data access across Netflix, enabling domain experts to build sophisticated data products without specialized stream processing knowledge. The second part of our presentation showcases these innovations in action through the Games Analytics Platform case study. As Netflix ventured into games, our Games Data team…

In this session, Sujay Jain from Netflix will reveal how a massive Data Mesh—powering over 20,000 Apache Flink jobs and thousands of Kafka topics—continues to evolve and spark cutting-edge innovation.

As data volume and velocity continue to grow at unprecedented rates, organisations are increasingly turning to stream processing to unlock real-time insights and fuel data-driven decision-making. This presentation will explore how Netflix has evolved its Data Mesh platform, a distributed data architecture, to embrace the power of Streaming SQL. The session will showcase the journey from the initial implementation of Data Mesh as a data movement platform to its transformation into a comprehensive stream processing powerhouse with the integration of the Data Mesh SQL Processor. Attendees will learn about the challenges Netflix faced with its earlier system, which relied on pre-built processors and the low-level Flink DataStream API, leading to limitations in expressiveness and a steep learning curve for developers. The presentation will then unveil the innovative solution: the Data Mesh SQL Processor, a platform-managed Flink job that harnesses the familiarity and versatility of SQL to simplify stream processing. Through practical examples, attendees will discover how the SQL Processor interacts with Flink's Table API, converting data streams into dynamic tables for seamless SQL…