Greg Harris

If you’ve been thinking to yourself “How do I actually use KIP-1150?” you’re not alone. Someone has already shown you a picture of the transformative nature of Diskless Topics: Cheap, Scalable, High-throughput, and Easy-to-operate topics, with a trade-off of higher latency. Maybe you’ve read the KIP, and figured out how it delivers on its technical promises. Now you’re on-board with rolling it out in your organization, but the instruction manual to do so is missing. That’s what we’re here to share. This talk will describe a Diskless Technical Quality Program that you can implement to get your organization ready for Diskless Topics, and your applications onboarded. We’ll cover the basic prerequisites such as KRaft and Tiered Storage, and how to size and configure your Diskless infrastructure for your workloads. After that, we’ll discuss the mechanics of onboarding applications to Diskless with a phased roll-out, building confidence in the process and encouraging more adoption over time. Finally, we’ll talk about how to handle architecture limitations, unplanned downtime, and communicating with leadership to keep everyone informed. With this tool kit you can customize the quality…

KIP-1150: Diskless Topics promises to make Apache Kafka more cost effective and flexible than ever before, but how does it work? Where does the cost savings come from? Is it really Diskless? What about Leaderless? Why is the latency worse? This talk will walk through the design for the preferred implementation in KIP-1163: Diskless Core, and answer all of these questions. A basic understanding of Apache Kafka is enough to attend this talk: we’ll review the architecture used for classic and tiered topics, and how data is produced and fetched. We'll discuss the limitations of this architecture in the context of modern hyperscaler cloud deployments, and where the costs become excessive. Then we’ll show how the basic components of Kafka are taken apart and reassembled to build the Diskless architecture. We’ll also discuss the major rejected alternatives, and compare KIP-1163 to similar KIPs working to solve the same problem. At the end of this session, you should feel confident talking to stakeholders and community members about this amazing upcoming feature! By attending this session, your contact information may be shared with the sponsor for relevant follow up for this event only.

Explore the past achievements, present realities, and future possibilities of Kafka/Kafka Streams with a mix of community leaders, contributors, and Apache Kafka committers. This is a panel so audience interaction is central and encouraged! Bring us your questions, challenge assumptions, and help shape the future of Kafka/Kafka Streams with this session

KIP-1150: Diskless Topics promises to make Apache Kafka more cost effective and flexible than ever before, but how does it work? Where does the cost savings come from? Is it really Diskless? What about Leaderless? Why is the latency worse? This talk will walk through the design for the preferred implementation in KIP-1163: Diskless Core, and answer all of these questions. A basic understanding of Apache Kafka is enough to attend this talk: we’ll review the architecture used for classic and tiered topics, and how data is produced and fetched. We'll discuss the limitations of this architecture in the context of modern hyperscaler cloud deployments, and where the costs become excessive. Then we’ll show how the basic components of Kafka are taken apart and reassembled to build the Diskless architecture. We’ll also discuss the major rejected alternatives, and compare KIP-1163 to similar KIPs working to solve the same problem. At the end of this session, you should feel confident talking to stakeholders and community members about this amazing upcoming feature!

Software becomes obsolete when it is surpassed by competitors that innovate. For the Apache Kafka’s project, innovative forks and re-implementations are together threatening to unseat it as the leader of the ecosystem.