Tyler Michael Smith

Meet the core maintainers of vLLM at this session! Come and discuss use cases, features, roadmap with us, or just learn how the vLLM development happens under the hood.

WideEP—wide expert parallelism fails not because experts are expensive, but because routing ignores where state already lives. In PyTorch LLM serving with vLLM, WideEP fans tokens across many experts while KV caches accumulate unevenly across data-parallel replicas. When routing is unaware of KV...

Beyond One-Size-Fits-All: Building Domain-Specific Language Models With PyTorch - Mayank Kumar, DeepTempo Double LLM RAG Accuracy by Adding GNNs - Serge Panev, NVIDIA Serving Transformers at Scale: KV-Cache-Aware Routing for PyTorch Inference - Maroon Ayoub, IBM & Tyler Michael Smith, Red Hat...