Navneeth Nair

A factory detects an equipment anomaly and needs the alert in seconds. A financial platform needs to process a transaction, enrich it with account history, and flag it for fraud—all before the customer closes the app. These workloads both involve streaming data, but they have very different requirements under the hood. This session introduces the two core streaming capabilities on Databricks, Real-Time Mode and Zerobus Ingest, and how to match them to your workload. Real-Time Mode on Apache Spark™ Structured Streaming is for when data needs to be acted upon, not just stored. With sub-second, event-driven computation and expressive stateful processing via TransformWithState, it's the right choice for operational workloads where latency directly affects business outcomes: live betting platforms, ad attribution pipelines, fraud detection, and real-time personalization. Zerobus Ingest is a Kafka-free ingestion capability that lets applications and devices write event data directly into Delta Lake, no message broker, no compute overhead, governed delivery out of the box. It's designed to move data fast and at scale. It does not process or transform data in flight, and for…

The data world isn't slowing down—it’s streaming faster than ever. And if you're still thinking batch-first, it's time to flip the script. Join us for an overview of new features in Apache Spark™ Structured Streaming. Learn how it is enabling AI workloads for Cursor, machine learning workflows in Coinbase, and complex real-time requirements of MakeMyTrip. If you are an advanced user, this session is for you: we will showcase advanced features like real-time mode, transform with state, and streaming from hybrid sources. If you like simplicity, this session is also for you: you will discover how Spark Declarative Pipelines’s constructs can simplify stream processing.

Data classification, extraction, and analysis can be challenging for organizations that deal with volumes of documents. Traditional document processing solutions are manual, expensive, error-prone, and difficult to scale.