Uploaded November 2024 | Updated September 2026, 3 weeks ago
Title: PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation
Speaker: Jason Ansel (Meta AI)
Abstract: PyTorch 2 leverages compilers to deliver faster training and inference without sacrificing the usability and flexibility for which PyTorch is known. It is fully backward compatible and continues to provide an imperative, extensible, easy-to-debug, and Pythonic programming environment for AI researchers, data scientists, and engineers. PyTorch 2 achieves a 2.27x speedup in inference and a 1.41x speedup in training (geometric mean) on an NVIDIA A100 GPU across 180+ real-world models, outperforming six other compilers.
This talk will explore the key technologies behind PyTorch 2: TorchDynamo and TorchInductor.
TorchDynamo is a Python-level JIT compiler designed to accelerate unmodified PyTorch programs. It hooks into the CPython frame evaluation API to dynamically modify Python bytecode just before execution. When rewriting bytecode, it extracts sequences of PyTorch operations into an FX Graph, which is then compiled with various extensible backends. TorchDynamo generates compiled graph fragments that can seamlessly mix with regular Python execution, giving better performance while maintaining the flexibility of Python.
TorchInductor, the default compiler backend in PyTorch 2, translates PyTorch programs into OpenAI's Triton for GPUs, OpenMP/C++ for CPUs, and now contains an experimental Halide backend targeting both. It maintains the flexibility and dynamism of PyTorch by employing abstractions similar to those in PyTorch’s eager mode. TorchInductor introduces a new define-by-run loop-level intermediate representation (IR), facilitating easy integration of new operator lowerings. Additionally, as it is implemented in Python, TorchInductor is straightforward for users to extend and modify to fit their specific needs.
Bio: Jason Ansel is a Principal Research Scientist at Meta AI and technical lead for PyTorch compilers. He started the TorchDynamo and TorchInductor projects, bringing flexible graph capture and a high-performance compiler to PyTorch 2. Jason holds a Ph.D. from MIT CSAIL, earned in 2014, with research focusing on the intersection of machine learning, compilers, and programming languages.
This video is closed captioned.
Title: PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation
Speaker: Jason Ansel (Meta AI)
Abstract: PyTorch 2 leverages compilers to deliver faster training and inference without sacrificing the usability and flexibility for which PyTorch is known. It is fully backward compatible and continues to provide an imperative, extensible, easy-to-debug, and Pythonic programming environment for AI researchers, data scientists, and engineers. PyTorch 2 achieves a 2.27x speedup in inference and a 1.41x speedup in training (geometric mean) on an NVIDIA A100 GPU across 180+ real-world models, outperforming six other compilers.
This talk will explore the key technologies behind PyTorch 2: TorchDynamo and TorchInductor.
TorchDynamo is a Python-level JIT compiler designed to accelerate unmodified PyTorch programs. It hooks into the CPython frame evaluation API to dynamically modify Python bytecode just before execution. When rewriting bytecode, it extracts sequences of PyTorch operations into an FX Graph, which is then compiled with various extensible backends. TorchDynamo generates compiled graph fragments that can seamlessly mix with regular Python execution, giving better performance while maintaining the flexibility of Python.
TorchInductor, the default compiler backend in PyTorch 2, translates PyTorch programs into OpenAI's Triton for GPUs, OpenMP/C++ for CPUs, and now contains an experimental Halide backend targeting both. It maintains the flexibility and dynamism of PyTorch by employing abstractions similar to those in PyTorch’s eager mode. TorchInductor introduces a new define-by-run loop-level intermediate representation (IR), facilitating easy integration of new operator lowerings. Additionally, as it is implemented in Python, TorchInductor is straightforward for users to extend and modify to fit their specific needs.
Bio: Jason Ansel is a Principal Research Scientist at Meta AI and technical lead for PyTorch compilers. He started the TorchDynamo and TorchInductor projects, bringing flexible graph capture and a high-performance compiler to PyTorch 2. Jason holds a Ph.D. from MIT CSAIL, earned in 2014, with research focusing on the intersection of machine learning, compilers, and programming languages.
This video is closed captioned.
![[Audio Descriptions] I Am CSE: Poojita Garg](https://i.ytimg.com/vi/ZVaEo4X1Mgk/mqdefault.jpg "[Audio Descriptions] I Am CSE: Poojita Garg
Poojita Garg is a PhD student in the Paul G. Allen School of Computer Science & Engineering. Her research is situated at the intersection of ubiquitous computing and health sensing systems. Poojita shares how she created an app that transforms off-the-shelf smartphones into fetal Doppler that functions without any external hardware, making crucial fetal health monitoring more widely available.
A version of this video without audio descriptions is available here: https://youtu.be/yUvXe-G2v-E")
