Uploaded September 2026 | Updated November 2026, 23 hours ago
"Github repository: github.com/anyscale/ray-summit-training-2026/tree/main/ray_summit_foundation_model_training_2026
Build and optimize large-scale distributed training systems for foundation models using modern parallelism and orchestration techniques.
This hands-on training session covers data, tensor, pipeline, sequence/context, and expert parallelism, multidimensional parallelism, DDP, FSDP, and DeepSpeed (ZeRO 1-3) training architectures, multi-node GPU scheduling and orchestration, checkpointing and fault recovery, distributed data sharding and ingestion with Ray Data, and training throughput and utilization optimization.
You'll leave knowing how systems built with Ray handle multi-node training, communication bottlenecks, and cluster-scale performance optimization.
Liked this video? Check out other Ray Summit training session recordings: youtube.com/playlist?list=PLUpYVYRTFCQM
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com/"
"Github repository: github.com/anyscale/ray-summit-training-2026/tree/main/ray_summit_foundation_model_training_2026
Build and optimize large-scale distributed training systems for foundation models using modern parallelism and orchestration techniques.
This hands-on training session covers data, tensor, pipeline, sequence/context, and expert parallelism, multidimensional parallelism, DDP, FSDP, and DeepSpeed (ZeRO 1-3) training architectures, multi-node GPU scheduling and orchestration, checkpointing and fault recovery, distributed data sharding and ingestion with Ray Data, and training throughput and utilization optimization.
You'll leave knowing how systems built with Ray handle multi-node training, communication bottlenecks, and cluster-scale performance optimization.
Liked this video? Check out other Ray Summit training session recordings: youtube.com/playlist?list=PLUpYVYRTFCQM
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com/"










