AI Training Parallelism, From a Network Engineers Point of View @TeamNANOG
AI Training Parallelism, From a Network Engineers Point of View  @TeamNANOG
Uploaded October 2026 | Updated November 2026, 12 hours ago
AI training runs across thousands of GPUs, and how you split the work decides what shows up on your network. This talk covers the main ways to split it : data, pipeline, tensor and mixture of experts. From a network engineer's point of view, not an ML one.

For each one: what it does to the model, what it puts on the wire, and what we do about it. Rail optimization, one-to-one subscription, lossless fabrics, load balancing, incast, all-to-all, and where scale-up stops and scale-out starts. Then how a real job runs all of them at once on the same wires.

The techniques come from the ML world. The network side comes from four years of building and running AI fabrics. If you build or run one, you will walk out knowing why your traffic looks the way it does, and how to design for it on purpose.

Adel Iazzag:
Adel Iazzag is a Senior Solutions Architect at NVIDIA working on networking for large-scale AI infrastructure. Before NVIDIA, he held network engineering roles at Oracle Cloud Infrastructure and AWS, working across datacenter networking, InfiniBand, RoCE, congestion control, and automation. He enjoys learning about large-scale network architectures and understanding how networking is evolving to support increasingly demanding AI workloads.
AI Training Parallelism, From a Network Engineers Point of ViewTutorial: Internet Interconnection and Observability WalkthroughNetwork Automation Pipelines – From Near-Zero to Hero in 45 MinutesShow Me Your PRsLarge Scale NAT Routing To Centralized Cluster over SRInside Google Peering: Trends, Recent Policy Changes, And Future DirectionWireshark - From Banned Application to an Everyday ToolNANOG 96 Members MeetingASPA La Vista.. Route LeaksNANOG 98 Conference OpeningNANOG 94 Conference OpeningVendor Independent Network Segmentation
NANOG |

AI Training Parallelism, From a Network Engineer's Point of View

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER