Uploaded October 2025 | Updated September 2026, 2 hours ago

---

Speaker: Charles Frye

The source code (in CuTe) for FlashAttention4 on Blackwell GPUs has recently been released for the forward pass. The following blog: modal.com/blog/reverse-engineer-flash-attention-4 goes over their findings when reading through the source code, and changes between FA1,2,3 and now 4!