Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives @GPUMODE

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives @GPUMODE

Uploaded August 2026 | Updated September 2026, 2 days ago

arxiv.org/abs/2607.16100

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives

Production Megakernels for Real-World Inference

Live Quartet 4 bit training

Lecture 24: Scan at the Speed of Light

Lecture 80: How FlashAttention 4 Works

Lecture 39: Torchtitan

Cornserve: Easy, Fast and Scalable Multimodal AI

Bonus Lecture: CUDA C++ llm.cpp

Lecture 35: SGLang

Lecture 69: Quartet 4 bit training

TIRx

Lecture 52: Scaling Laws for Low Precision

Lecture 113: Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives

🔇 TAP TO UNMUTE