Dheeraj Puli

Discover how National Australia Bank (NAB) scales production streaming with Spark Declarative Pipelines (SDP), using Streaming Tables and Materialized Views to enable rewind and replay — restoring pipelines to a consistent state, applying fixes safely and reprocessing only affected data. Explore how this approach reduces brittle manual interventions and avoids full refreshes by leaning on declarative semantics, governed data products, and robust checkpoint/versioning patterns and helps proactively prevent bad data from reaching downstream systems and ensures that data demographics can be safely restored to their previous usable state in the Lakehouse. Walk away with actionable designs, checklists and success criteria for resilient, cost‑efficient streaming at enterprise scale in regulated environments. Outcomes include faster recovery, lower operational effort and cost, improved reliability and correctness and sustained performance as concurrency grows.