How FlashAttention 4 Works @GPUMODE
How FlashAttention 4 Works @GPUMODE
Uploaded October 2025 | Updated September 2026, 10 hours ago
Speaker: Charles Frye

From the Modal team: modal.com/blog/reverse-engineer-flash-attention-4
How FlashAttention 4 WorksLecture 20: Scan AlgorithmLivestream: Outperforming cuBLAS on H100Mirage (MPK): Compiling LLMs into Mega KernelsLecture 114: PyCuTeLearning CUTLASS the hard wayLecture 63: Search-Based Deep Learning CompilersLecture 89: cuTile (from friends at NVIDIA)PyCuTeSpectral Compute: Compile CUDA everywhereLecture 68: Landscape of GPU Centric communicationMulti-GPU programming
GPU MODE |

How FlashAttention 4 Works

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER