Foundation Model Distributed Training with Ray | Anyscale | Ray Summit 2026 @anyscale
Foundation Model Distributed Training with Ray | Anyscale | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 23 hours ago
"Github repository: github.com/anyscale/ray-summit-training-2026/tree/main/ray_summit_foundation_model_training_2026

Build and optimize large-scale distributed training systems for foundation models using modern parallelism and orchestration techniques.

This hands-on training session covers data, tensor, pipeline, sequence/context, and expert parallelism, multidimensional parallelism, DDP, FSDP, and DeepSpeed (ZeRO 1-3) training architectures, multi-node GPU scheduling and orchestration, checkpointing and fault recovery, distributed data sharding and ingestion with Ray Data, and training throughput and utilization optimization.

You'll leave knowing how systems built with Ray handle multi-node training, communication bottlenecks, and cluster-scale performance optimization.

Liked this video? Check out other Ray Summit training session recordings: youtube.com/playlist?list=PLUpYVYRTFCQM

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com/"
Foundation Model Distributed Training with Ray | Anyscale | Ray Summit 2026High-Throughput Inference for Synthetic Data & Evals at Sutro | Ray Summit 2025How a Transformers Model Loads in vLLM | Hugging Face | Ray Summit 2026NVIDIA’s Framework for Scalable Data Curation | Ray Summit 2025Koba: Disaggregated Data Loading for Multimodal Training at Luma AI | Luma AI | Ray Summit 2026A Compiler for Christmas | Brendan Burns #raysummitTowards Clinically Grounded LLMs | SonderMind | Ray Summit 2026Scaling LLM Post-Training at Character.AI | Ray Summit 2025Webinar: Getting Started with Distributed Training at ScaleHighly Compressed Gemma Models with Custom Quantization-Aware Training | DeepMind | Ray Summit 2026Multi-Modal Data Processing Pipelines for AI Systems | Anyscale | Ray Summit 2026Ray Across the Stack Fine Tuning Qwen and Serving 3D Generation at Autodesk with AWS Trainium and In
Anyscale |

Foundation Model Distributed Training with Ray | Anyscale | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER