Derek Troy-West

On an average day, around 400 engineers across TD interact with our event streaming platform. Almost none of them sit on the platform team, and all of them work inside one of the most heavily regulated industries in North America. So how do we balance broad self-service against strict regulatory obligation, giving engineers free rein while meeting the auditing and access requirements that come with banking? That is the operational problem we set out to solve. This talk walks through how we built and operate a multi-tenant Kafka platform that treats governance as an enabler of self-service rather than a brake on it. We will cover the tenancy model that isolates teams across a shared cluster estate, role-based access control mapped to topics, consumer groups, and schemas, and the staged approval workflows that let us automate intake without losing the audit trail. We will go into data masking for sensitive payloads, the audit logging that turns regulatory compliance into a byproduct of normal operation rather than a separate effort, and the data catalog to operationalise metadata and give engineers a way to discover and understand the streams available to them. Throughout, we will…

$32 Tejansh Rana, Senior Principal Engineer, Autodesk Boyuan Huang, Product Director, Alibaba Cloud Platform for AI Derek Troy-West, CEO, Factor House Erik Schmiegelow, CEO, Hivemind Moderator: Lorenzo Affetti, Staff Software Engineer, Ververica