Laikh Tewari

The explosive growth of large language models demands new techniques to efficiently handle inference at scale. In this talk we'll show how you can easily extend your PyTorch programs with NVIDIA’s Dynamo and TensorRT-LLM to go faster and wider. Dynamo is a low-latency inference framework that works...

Purge the GIL: Improved Torch.DataLoader - Michal Szolucha & Rostan Tabet, NVIDIA XFormers - Daniel Haziza, Meta AI  TritonCC: AOT Triton Workflow for TorchScript C++ Runtime - Sijia Chen & Huamin Li, Meta The PyTorch 2.0 Inference Story - Angela Yi, Bin Bao, Sheng Qin & Sherlock Huang, Meta Tensor...