Wout de Ruiter

In this talk, we showcase how NS (Dutch Railways) built an end-to-end Databricks solution that processes train telemetry in real time, enabling live detection of carriage failures and improving overall reliability. The platform handles more than 100 billion data points per day with sub-second end-to-end latency. The session features a live demo of real train telemetry, giving the audience a direct, in‑room experience of real time. We explain how the solution was engineered using the new Real-Time Mode for Spark Structured Streaming. The entire stack is built in PySpark and SQL, so no microservices, no separate streaming framework, just Databricks. Throughout the session, we connect to other high velocity domains with cases like real time parcel rerouting, live airport baggage tracking and real-time signals feeding analytics and AI. Attendees leave with practical insights into bringing real time data into the modern data stack without the complexity of traditional streaming systems.