Benchmarking GPU Scheduling for Massive-Scale Ray Workloads at Minimal Cost - MSFT | Ray Summit 2025 @anyscale
Benchmarking GPU Scheduling for Massive-Scale Ray Workloads at Minimal Cost - MSFT | Ray Summit 2025  @anyscale
Uploaded December 2025 | Updated November 2026, 4 days ago
At Ray Summit 2025, Anson Qian from Microsoft shares a novel, cost-effective approach for benchmarking GPU scheduling and large-scale Ray workloads on Kubernetes—using Kwok (Kubernetes Without Kubelet) to simulate massive Ray clusters without requiring massive hardware.

He begins by outlining a key challenge in ML infrastructure development: validating the scalability and stability of Ray, KubeRay, and Kubernetes GPU scheduling typically requires extremely large and expensive clusters. Traditional testing at this scale is often cost-prohibitive, making it difficult for teams to evaluate performance under realistic production loads.

Anson introduces a breakthrough solution: using Kwok to simulate 10,000 RayCluster and RayJob workloads across 10,000 GPU nodes—all running on a modest, cloud-managed Kubernetes cluster, such as a 3-node Azure Kubernetes Service (AKS) setup. The entire benchmark environment runs for roughly $10 per hour, making large-scale infrastructure testing accessible to any team.

This session covers:

How Kwok simulates massive Kubernetes clusters without real GPU nodes

How to stress-test KubeRay, the Kubernetes control plane, and Ray’s scheduling logic at extreme scale

Techniques for validating stability, analyzing performance, and uncovering bottlenecks before real-world deployment

Why this simulated approach accelerates the development of robust, production-grade ML systems

Anson demonstrates how this simulation framework provides a practical, low-cost path to evaluating large-scale ML infrastructure—unlocking a level of testing depth previously accessible only to teams with enormous compute budgets.

Attendees will leave with an actionable blueprint for scaling Ray, testing GPU scheduling logic, and validating ML infrastructure in a highly efficient, affordable way.

Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh

Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
Benchmarking GPU Scheduling for Massive-Scale Ray Workloads at Minimal Cost - MSFT | Ray Summit 2025Embedded LLM’s Guide to vLLM Architecture & High-Performance Serving | Ray Summit 2025Ray Agent Engine: Deploying AI Agents with Ray Serve | Ray Summit 2025Planetary-scale multimodal image processing | Xoople + AnyscalePlatform Orchestration: Anyscale + Ray on Kubernetes | Anyscale | Ray Summit 2026How DataRobot Parallelizes Agentic Pipeline Searches with Ray + syftr | Ray Summit 2025Diffusion Large Language Models | Inception Labs | Ray Summit 2026Efficient, High-Performance AI Inferencing with Intel Xeon 6 | Ray Summit 2025Cascade: Write-Once Run-Anywhere Inference on Ray | Samsara | Ray Summit 2026Maximizing GB200 and GB300 Performance with Domain-Aware Scheduling | NVIDIA | Ray Summit 2026Ray Data on GPU - Breaking Preprocessing Bottleneck for Model Training | Stripe | Ray Summit 2026Miles: Enterprise-Facing Agentic RL Framework | RadixArk + Anyscale | Ray Summit 2026
Anyscale |

Benchmarking GPU Scheduling for Massive-Scale Ray Workloads at Minimal Cost - MSFT | Ray Summit 2025

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER