Gaurav Narula

Authenticating users is crucial in every production Kafka deployment. Apache Kafka ships with diverse authentication options, including password based SASL mechanisms and mTLS. As computing workloads adopt identities in the form of short-lived X.509 certificates, using them for mTLS offers significant advantages over passwords as they limit the impact of a credential leak and cannot be brute-forced. This talk starts by looking into how authentication works in Kafka and different configurations to customise it. We'll cover challenges faced when migrating users to mTLS and review options to minimise the operational effort. Then, we will share an approach that adds support for mTLS on the SASL listener so users can continue using their existing KafkaPrincipal and fallback to passwords seamlessly during the migration, giving cluster administrators and users confidence before moving away from SASL. Finally we will talk about how enabling Kafka brokers to serve distinct server and client certificates supports adoption of mTLS for inter-broker communication, and the learnings and pitfalls of rolling this out in the fleet.