Uploaded November 2025 | Updated November 2026, 11 hours ago
At Ray Summit 2025, Spencer Peterson and Raja Jadeja from Google share the definitive playbook for using KubeRay to build a stable, efficient, and self-service AI platform on Kubernetes—eliminating the operational chaos caused by ad-hoc ML deployments, YAML overload, and runaway cluster resource usage.
They begin by calling out the common challenges faced by ML infrastructure teams:
Unschedulable pods when a single rogue job consumes all available resources
Idle GPUs inflating cloud bills
Developers trapped in complex Kubernetes configs just to scale a Python script
Manual deployments that are brittle, slow, and impossible to manage at scale
KubeRay solves these issues by transforming Kubernetes into an application-aware backend for distributed AI. With the RayCluster custom resource, Kubernetes can natively manage Ray applications end-to-end—from creation and scheduling to autoscaling, fault tolerance, and cleanup.
Spencer and Raja walk through the operational strategies needed to run production Ray clusters effectively:
Tame the noisy neighbor problem: Implement true multi-tenancy using namespace isolation and cgroups to ensure resource fairness.
Master workload placement: Use Ray’s placement features and Kubernetes labels to automatically route jobs to the optimal nodes—maximizing efficiency without requiring deep Kubernetes expertise.
Cut cloud costs with smart scaling: Configure IPPR with the Ray v2 autoscaler for faster pod scheduling and on-demand scaling, eliminating costly over-provisioning.
Run a unified multi-cluster platform: Learn best practices for managing multiple specialized Ray clusters under one Kubernetes control plane.
Attendees will leave with a clear, actionable blueprint for turning Kubernetes into a robust, cost-efficient, developer-friendly platform for distributed AI—powered by KubeRay.
Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh
Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
At Ray Summit 2025, Spencer Peterson and Raja Jadeja from Google share the definitive playbook for using KubeRay to build a stable, efficient, and self-service AI platform on Kubernetes—eliminating the operational chaos caused by ad-hoc ML deployments, YAML overload, and runaway cluster resource usage.
They begin by calling out the common challenges faced by ML infrastructure teams:
Unschedulable pods when a single rogue job consumes all available resources
Idle GPUs inflating cloud bills
Developers trapped in complex Kubernetes configs just to scale a Python script
Manual deployments that are brittle, slow, and impossible to manage at scale
KubeRay solves these issues by transforming Kubernetes into an application-aware backend for distributed AI. With the RayCluster custom resource, Kubernetes can natively manage Ray applications end-to-end—from creation and scheduling to autoscaling, fault tolerance, and cleanup.
Spencer and Raja walk through the operational strategies needed to run production Ray clusters effectively:
Tame the noisy neighbor problem: Implement true multi-tenancy using namespace isolation and cgroups to ensure resource fairness.
Master workload placement: Use Ray’s placement features and Kubernetes labels to automatically route jobs to the optimal nodes—maximizing efficiency without requiring deep Kubernetes expertise.
Cut cloud costs with smart scaling: Configure IPPR with the Ray v2 autoscaler for faster pod scheduling and on-demand scaling, eliminating costly over-provisioning.
Run a unified multi-cluster platform: Learn best practices for managing multiple specialized Ray clusters under one Kubernetes control plane.
Attendees will leave with a clear, actionable blueprint for turning Kubernetes into a robust, cost-efficient, developer-friendly platform for distributed AI—powered by KubeRay.
Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh
Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










