Uploaded September 2026 | Updated November 2026, 2 days ago
ML evaluation pipelines have three stages that run at different speeds, and running them together on shared resources starves GPUs down to 10 to 40 percent utilization. Motive's rebuilt pipeline runs above 90 percent, and eval iterations that skip the GPU entirely got a 200x speedup.
At Ray Summit 2026, Tim Cheng and Hareesh Kolluru from Motive show how they rebuilt the pipeline on Ray with each stage on its own tier, CPU-heavy work on wide cheap instances and GPU inference on a separately sized tier, with Ray Data expressing the whole pipeline as one dataflow. Disaggregation creates an explicit interface between the GPU stage and downstream CPU stages, making inference outputs cacheable: the roughly 95 percent of iterations in an active eval cycle that only change scoring logic skip the GPU and replay from cache, turning a multi-hour GPU-bound regression into a CPU-bound replay in minutes.
You'll leave with code for diagnosing GPU feeder starvation, expressing two-dimensional parallelism with Ray Data, and identifying cacheable boundaries in heterogeneous pipelines.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
ML evaluation pipelines have three stages that run at different speeds, and running them together on shared resources starves GPUs down to 10 to 40 percent utilization. Motive's rebuilt pipeline runs above 90 percent, and eval iterations that skip the GPU entirely got a 200x speedup.
At Ray Summit 2026, Tim Cheng and Hareesh Kolluru from Motive show how they rebuilt the pipeline on Ray with each stage on its own tier, CPU-heavy work on wide cheap instances and GPU inference on a separately sized tier, with Ray Data expressing the whole pipeline as one dataflow. Disaggregation creates an explicit interface between the GPU stage and downstream CPU stages, making inference outputs cacheable: the roughly 95 percent of iterations in an active eval cycle that only change scoring logic skip the GPU and replay from cache, turning a multi-hour GPU-bound regression into a CPU-bound replay in minutes.
You'll leave with code for diagnosing GPU feeder starvation, expressing two-dimensional parallelism with Ray Data, and identifying cacheable boundaries in heterogeneous pipelines.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










