Pinak Panigrahi

Can you train an LLM on AWS Trainium, our purpose-built AI chip for better price performance, using PyTorch with minimal code changes? We'll show you with nanochat, a 560M-parameter GPT you can train in about two hours on a single Trainium instance using familiar PyTorch workflows.

Every team moving AI to production hits the same wall: scaling frontier model inference against growing token demand, tighter latency budgets, and spiraling compute costs. This workshop tackles that problem head-on.

In this interactive chalk talk, we'll collaboratively explore how to optimize LLM serving using vLLM on AWS Trainium and Inferentia chips. Through whiteboarding and hands-on discussion, we'll architect efficient deployment patterns, tackle common performance bottlenecks, and design scalable serving configurations.

Join us for a immersive, hands-on workshop to learn to deploy and optimize large language models (LLMs) for scalable inference at enterprise scale. Participants will learn to orchestrate distributed LLM serving with vLLM on Amazon EKS, enabling robust, flexible, and highly available deployments.

Large language models (LLMs) show remarkable impact in a wide range of applications, but they pose significant cost and deployment challenges at scale. In this workshop, explore best practices for efficient LLM deployment using AWS Inferentia2-based Amazon EC2 Inf2 instances to maximize performance and lower costs.