Lecture 17: NCCL @GPUMODE

Lecture 17: NCCL @GPUMODE

Uploaded May 2024 | Updated September 2026, 2 hours ago

Code and Slides: github.com/cuda-mode/lectures/tree/main/lecture_017

Lecture 17: NCCL

Lecture 96: TLX

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives

Production Megakernels for Real-World Inference

Live Quartet 4 bit training

Lecture 24: Scan at the Speed of Light

Lecture 80: How FlashAttention 4 Works

Lecture 39: Torchtitan

Cornserve: Easy, Fast and Scalable Multimodal AI

Bonus Lecture: CUDA C++ llm.cpp

Lecture 35: SGLang

Lecture 69: Quartet 4 bit training

Lecture 17: NCCL

🔇 TAP TO UNMUTE