Simon Aubury

In aviation, real-time data keeps flights safe, aircraft moving, and operations running smoothly. In this talk, we’ll explore how Kafka-based streaming powers aviation - from orchestrating fast aircraft turnarounds on the ground, to monitoring flight performance in the air, and enabling instant decisions by crew and ground teams through connected operational systems. Drawing on real-world experience building an airline-scale streaming platform, I’ll share practical lessons for platform engineers, including: - Designing for failure, not perfection - making failures predictable, contained, and recoverable through idempotence, DLQs, and retry strategies - Managing transformations at scale – ksqlDB patterns and lessons learned handling complex XML payloads - Isolating workloads through tenant separation - providing streaming corridors for data, compute, and fault containment - Enforcing data contracts - managing schema evolution across disparate aviation operational systems - Keeping it simple in complex environments – building boring, understandable, and debuggable pipelines You’ll leave with practical patterns and mental models for building Kafka based streaming platforms that are…

Got a billion rows of data in a weird file format? Wishing you could wrangle a dataframe from a geospatial dataset? A bit lost interacting with a remote API? Let’s wrangle some data with Python and DuckDB. DuckDB executes analytical SQL queries without the need for a server. DuckDB features a...