Lecture 8: CUDA Performance Checklist @GPUMODE

Lecture 8: CUDA Performance Checklist @GPUMODE

Uploaded March 2024 | Updated September 2026, 2 hours ago

Code github.com/cuda-mode/lectures/tree/main/lecture8
Slides docs.google.com/presentation/d/1cvVpf3ChFFiY4Kf25S4e4sPY6Y5uRUO-X-A4nJ7IhFE/edit
Studio lightning.ai/msaroufim/studios/cuda-mode-lectures?section=featured&query=cuda+mode

Lecture 8: CUDA Performance Checklist

Lecture 16: On Hands Profiling

Lecture 17: NCCL

Lecture 96: TLX

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives

Production Megakernels for Real-World Inference

Live Quartet 4 bit training

Lecture 24: Scan at the Speed of Light

Lecture 80: How FlashAttention 4 Works

Lecture 39: Torchtitan

Cornserve: Easy, Fast and Scalable Multimodal AI

Bonus Lecture: CUDA C++ llm.cpp

Lecture 8: CUDA Performance Checklist