Matt Slack

Checkout.com is a leading global digital payments provider, processing over $300 billion in e-commerce volume annually. Our fraud models depend on fresh features, but batch pipelines limited how quickly we could detect new fraud patterns. In this session, we’ll show how we built real-time feature pipelines using Spark Structured Streaming on Databricks, reducing feature computation to a few seconds and unlocking an eight-figure revenue opportunity. We’ll cover our technical approach, including custom stateful aggregations with transformWithStateInPandas, event-time watermarking for late payments, and TTL-based state management for deduplication. We’ll also discuss cost optimization at billion-event scale, and consistency between online and offline features. By incorporating real-time features into our fraud models, we achieved a 2% increase in PR-AUC, translating into millions in prevented fraud losses - offering a practical blueprint for real-time machine learning at scale.

Barclays Post Trade real-time trade monitoring platform was historically built on a complex set of legacy technologies including Java, Solace, and custom micro-services.This session will demonstrate how the power of Lakeflow Declarative Pipelines' new real-time mode, in conjunction with the foreach_batch_sink, can enable simple, cost-effective streaming pipelines that can load high volumes of data into Databricks new Serverless OLTP database with very low latency.Once in our OLTP database, this can be used to update real-time trading dashboards, securely hosted in Databricks Apps, with the latest stock trades - enabling better, more responsive decision-making and alerting.The session will walk-through the architecture, and demonstrate how simple it is to create and manage the pipelines and apps within the Databricks environment.