Balaji Varadarajan

We will start with two diagnostic tools: Little’s Law which translates your QPS and target latency into the concurrency and batch size your system actually needs and the roofline model which tells you whether prefill (compute-bound) or decode (memory-bound) is your real constraint before you reach for a fix..

In the age of GenAI, it’s tempting to chase rapid prototypes and flashy demos—but real business value lies in building sustainable, scalable systems that work beyond the lab. In this talk, we’ll cut through the hype and dive deep into the MLOps infrastructure required to support production-ready GenAI applications at scale.