Heterogeneous Pipelines as the Unlock for ML at Scale | Motive | Ray Summit 2026 @anyscale
Heterogeneous Pipelines as the Unlock for ML at Scale | Motive | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 2 days ago
ML evaluation pipelines have three stages that run at different speeds, and running them together on shared resources starves GPUs down to 10 to 40 percent utilization. Motive's rebuilt pipeline runs above 90 percent, and eval iterations that skip the GPU entirely got a 200x speedup.

At Ray Summit 2026, Tim Cheng and Hareesh Kolluru from Motive show how they rebuilt the pipeline on Ray with each stage on its own tier, CPU-heavy work on wide cheap instances and GPU inference on a separately sized tier, with Ray Data expressing the whole pipeline as one dataflow. Disaggregation creates an explicit interface between the GPU stage and downstream CPU stages, making inference outputs cacheable: the roughly 95 percent of iterations in an active eval cycle that only change scoring logic skip the GPU and replay from cache, turning a multi-hour GPU-bound regression into a CPU-bound replay in minutes.

You'll leave with code for diagnosing GPU feeder starvation, expressing two-dimensional parallelism with Ray Data, and identifying cacheable boundaries in heterogeneous pipelines.

Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Heterogeneous Pipelines as the Unlock for ML at Scale | Motive | Ray Summit 2026Building a Virtual Biology Platform on Ray | Recursion | Ray Summit 2026Ray Joins The Linux Foundation & PyTorch Sub-Foundation: Toward a Unified AI Compute StackHow Torc Robotics Scales Multimodal AI for Autonomous Driving with RayHow the VAST AI Operating System Powers a Dynamic Data Plane for Ray | Ray Summit 2025Tailor-Made Inference at Scale | Simplismart | Ray Summit 2026Operating a Production-Scale Physical AI Data Platform with Ray | Hyundai | Ray Summit 2026Serving LLMs for Agents at Scale with Ray + vLLM | JPMorgan Chase | Ray Summit 2026Lessons from Unifying Simulation and RL Training | Torc Robotics | Ray Summit 2026Ray Train: Distributed Solutions for Removing Training Bottlenecks | Ray Summit 2025Building a Multimodal Video Processing Pipeline with RayHow Open-Source vLLM Topped the Artificial Analysis Leaderboard | DigitalOcean | Ray Summit 2026
Anyscale |

Heterogeneous Pipelines as the Unlock for ML at Scale | Motive | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER