Uploaded August 2026 | Updated November 2026, 2 days ago
This is the live presentation of Resilient AI Supercomputer Networking using MRC and SRv6 at the Santa Clara Convention Center.
Tail latency dominates the performance of synchronous pretraining jobs when running at very large scales. We describe a three-pronged approach: (1) a new RDMA-based transport protocol, MRC, sprays across many paths and actively load-balances between them, eliminating the issue of flow collisions and (2) the use of static source-routing using SRv6 to allow MRC the freedom to bypass failures by itself.
We describe our experiences running MRC and static SRv6 routing in production in OpenAI, where it has been used to train frontier models. We demonstrate how MRC allows AI training jobs to ride out many network failures that previously would have interrupted training.
Learn more about the @Scale conferences here: atscaleconference.com
This is the live presentation of Resilient AI Supercomputer Networking using MRC and SRv6 at the Santa Clara Convention Center.
Tail latency dominates the performance of synchronous pretraining jobs when running at very large scales. We describe a three-pronged approach: (1) a new RDMA-based transport protocol, MRC, sprays across many paths and actively load-balances between them, eliminating the issue of flow collisions and (2) the use of static source-routing using SRv6 to allow MRC the freedom to bypass failures by itself.
We describe our experiences running MRC and static SRv6 routing in production in OpenAI, where it has been used to train frontier models. We demonstrate how MRC allows AI training jobs to ride out many network failures that previously would have interrupted training.
Learn more about the @Scale conferences here: atscaleconference.com










