Unlocking Peak Workload Performance & Efficiency with Ray on Kubernetes | Ray Summit 2025 @anyscale
Unlocking Peak Workload Performance & Efficiency with Ray on Kubernetes | Ray Summit 2025  @anyscale
Uploaded November 2025 | Updated November 2026, 12 hours ago
At Ray Summit 2025, Nisha Mariam Johnson and Ryan O'Leary from Google share how to build an efficient, cost-effective, and accelerator-aware ML platform using Ray on Kubernetes—designed to tackle challenges in scheduling, cloud costs, and emerging hardware.

They begin by outlining how teams can establish a smarter foundation for resource management. This includes mastering GKE Custom Compute Classes and label-based scheduling to seamlessly shift workloads across spot instances, diverse GPU and TPU types, and reserved capacity—dramatically reducing cloud costs while maintaining strong performance. They also cover how GKE image streaming speeds up image and model loading to reduce startup latency.

Next, Nisha and Ryan explain how fair-share scheduling is achieved with Kueue, Kubernetes’ native job-queueing system. Integrated directly with Ray, Kueue ensures teams receive fair access to compute resources for both training and serving workloads, preventing starvation and improving reliability across shared clusters.

With this foundation in place, the speakers dive into how these capabilities support high-value ML workloads:

Large-scale training: Learn how to train on TPUs using Ray Train with familiar frameworks like JAX and PyTorch, and how integrated TPU metrics in the Ray Dashboard make performance debugging dramatically simpler.

High-throughput serving and batch inference: See how workloads can be dynamically shifted between GPUs and TPUs to optimize for both cost and latency. They also preview upcoming Ray features that further enhance large-scale training, high-throughput inference, and complex RL and agentic workloads—all running on a secure, decoupled GKE architecture.

Attendees will leave with a clear blueprint for the recommended architecture for Ray on Kubernetes, unlocking new levels of performance, cost-efficiency, and operational simplicity for ML workloads of any scale.

Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Unlocking Peak Workload Performance & Efficiency with Ray on Kubernetes | Ray Summit 2025Boosting vLLM Inference on Huawei NPU with Ray Compiled Graphs — Huawei | Ray Summit 2025Simulating the Physical World at Scale: Humanoid Locomotion with Ray | Lambda | Ray Summit 2026KV Cache and Long-Context Inference: Solutions for Inference at Scale | VAST Data | Ray Summit 2026Reward Hacking and the Gaps in Agentic AI | Ion Stoica (Anyscale) | Ray Summit 2026How AWS Scales Reinforcement Learning Across Thousands of GPUs | Ray Summit 2025Distributed Model Training with Ray at Capital One | Ray Summit 2025State of vLLM 2026 | Inferact | Ray Summit 2026From Benchmark #1 to Production | Nebius | Ray Summit 2026Foundation Model Distributed Training with Ray | Anyscale | Ray Summit 2026High-Throughput Inference for Synthetic Data & Evals at Sutro | Ray Summit 2025How a Transformers Model Loads in vLLM | Hugging Face | Ray Summit 2026
Anyscale |

Unlocking Peak Workload Performance & Efficiency with Ray on Kubernetes | Ray Summit 2025

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER