Bill Bejeck

For years, exactly-once Kafka Streams applications have carried a hidden tax: when a task crashes or a transaction times out, Streams can't trust its local state, so it wipes every state store and rebuilds it from the changelog topic. For large stores, that "wipe-and-rebuild" can take hours — or even days — turning a momentary blip into a prolonged outage. This talk explains how two KIPs working in tandem eliminate that tax. KIP-1035 moves changelog offsets out of the fragile external .checkpoint file and into the state store itself, persisting them atomically alongside the data in a dedicated RocksDB column family. KIP-892 builds on that foundation, buffering writes and committing data and offsets in a single atomic operation. On failure, the uncommitted buffer is simply discarded — the on-disk state already matches the last committed offset, so there is nothing to rebuild. We'll go under the hood: discussing new APIs, atomic flush, read-your-writes isolation levels, new configurations for memory-pressure control, and what all this means for custom state store implementations. The payoff is concrete and operational. You'll learn how recovery time drops from hours to seconds and…

Explore the past achievements, present realities, and future possibilities of Kafka/Kafka Streams with a mix of community leaders, contributors, and Apache Kafka committers. This is a panel so audience interaction is central and encouraged! Bring us your questions, challenge assumptions, and help shape the future of Kafka/Kafka Streams with this session

Picture this: your Kafka application is humming along perfectly in development, but in production, throughput tanks and latency spikes. The culprit? That "simple" serialization choice you made without much thought. What seemed like a minor technical detail just became your biggest bottleneck. Every Kafka record—whether flowing through KafkaProducer, KafkaConsumer, Streams, or Connect—must be converted to bytes over TCP connections. This serialization step occupies a tiny footprint in your code but wields outsized influence over your application's performance. For Kafka Streams stateful operations, this impact multiplies as records serialize and deserialize on every state store access. You could grab a serializer that ships with Kafka and call it done. But depending on your data structure and use patterns, the wrong choice can cost you critical performance. The right choice can transform your application from sluggish to lightning-fast. This talk dives deep into serialization performance comparisons across different scenarios. We'll explore critical trade-offs: the governance and evolution benefits of Schema Registry versus the raw speed of high-performance serializers. You'll see…

Stream processing has become the standard way to work with event data with Apache Kafka®, Kafka Streams, and Apache Flink® being the top choices for implementing an event streaming application. In many cases, an individual event isn't the focus of the streaming applications.