Harshada Yesane

At large scale, reliability is unforgiving. When a data platform processes trillions of events per day, even small delays or inconsistencies can ripple across analytics, AI systems, and customer-facing products. In these environments, Change Data Capture (CDC) pipelines are no longer just ingestion tools — they become core production infrastructure with strict latency and correctness requirements. In this talk, we’ll share lessons from operating Brooklin, an open-source data streaming platform used at LinkedIn to run reliable CDC and Kafka mirroring pipelines at massive scale. Brooklin processes over 7 trillion messages per day across 50+ clusters, mirrors 100k+ Kafka topics, and supports sub-minute SLAs for critical workloads spanning multiple teams and use cases. Rather than focusing on how to build CDC systems from scratch, this session emphasizes how platform teams can adopt proven patterns to operate CDC and Kafka mirroring reliably in real-world environments. We’ll discuss common CDC use cases across database-heavy organizations, including capturing changes from systems such as MySQL, Oracle, and TiDB, streaming them into Apache Kafka, and mirroring data across Kafka…