Pratyusha Singaraju

Production pipelines combine models, rule-based automation, and human reviewers to achieve a single outcome. Designing systems that allow these to coexist reliably at scale is a hard problem. At Netflix, every title that reaches our millions of subscribers passes through complex content metadata workflows. Some decisions are fully automated by rules. Others are handled by ML models. And some require a human to make the call. Building a system that is data-driven and coordinates all of these consistently — without breaking when any one component fails, changes, or gets replaced — required rethinking how we architect workflow orchestration entirely. In this talk, we share how we built a production framework where rule-based automation, ML models, and human-in-the-loop review are all first-class citizens in the same pipeline. We'll cover how we handle routing, failure isolation, and feedback across heterogeneous components — and how this foundation naturally extends to incorporate AI agents in the future.