Horace He

Introducing a novel abstraction leveraging the PyTorch compiler stack to enable custom, user-defined attention mechanisms. This new API supports dynamic modifications to attention scores within SDPA, providing both runtime and memory efficiency through kernel fusion with the FlashAttention...

Activation checkpointing is a commonly used technique to reduce memory usage during model training by reducing the number of activations saved for backward. Instead of keeping tensors needed for backward alive until they are used in gradient computation during backward, those tensors are recomputed...

Purge the GIL: Improved Torch.DataLoader - Michal Szolucha & Rostan Tabet, NVIDIA XFormers - Daniel Haziza, Meta AI  TritonCC: AOT Triton Workflow for TorchScript C++ Runtime - Sijia Chen & Huamin Li, Meta The PyTorch 2.0 Inference Story - Angela Yi, Bin Bao, Sheng Qin & Sherlock Huang, Meta Tensor...

There is a Cambrian explosion of performant and efficient methods to train and serve generative AI models within the community. The PyTorch team will present optimizations to transformer based Generative AI models, using pure, native PyTorch. In this talk we aim to cover both new techniques in...