Uploaded September 2026 | Updated November 2026, 12 hours ago
Teams training large models on Ray often run three separate footprints: one for data processing, one for distributed training, one for serving. Each brings its own scheduler, its own scaling behavior, and its own idle GPUs.
At Ray Summit 2026, Shreyas Adiyodi and Anoop Saha from AWS walk through collapsing that into a single Amazon EKS cluster with SageMaker HyperPod: scheduling heterogeneous CPU and GPU Ray clusters against shared capacity with queue-based admission, where checkpointing belongs when a run spans days, recovering from node-level hardware failure without restarting the pipeline, tightening the experimentation loop, and deploying models for accelerated inference.
You'll leave with a reference architecture for end-to-end Ray pipelines on Kubernetes that maximizes GPU utilization across every stage of model development.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Teams training large models on Ray often run three separate footprints: one for data processing, one for distributed training, one for serving. Each brings its own scheduler, its own scaling behavior, and its own idle GPUs.
At Ray Summit 2026, Shreyas Adiyodi and Anoop Saha from AWS walk through collapsing that into a single Amazon EKS cluster with SageMaker HyperPod: scheduling heterogeneous CPU and GPU Ray clusters against shared capacity with queue-based admission, where checkpointing belongs when a run spans days, recovering from node-level hardware failure without restarting the pipeline, tightening the experimentation loop, and deploying models for accelerated inference.
You'll leave with a reference architecture for end-to-end Ray pipelines on Kubernetes that maximizes GPU utilization across every stage of model development.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










