Ray Agent Engine: Deploying AI Agents with Ray Serve | Ray Summit 2025 @anyscale
Ray Agent Engine: Deploying AI Agents with Ray Serve | Ray Summit 2025  @anyscale
Uploaded December 2025 | Updated November 2026, 18 hours ago
At Ray Summit 2025, Juncheng Ma, Deepak Chandramouli, and Bhumik Thakkar from Apple share how Ray Serve can be used as a powerful, framework-agnostic foundation for deploying the next generation of AI agents at scale.

They begin by outlining the growing complexity of modern AI agents—spanning multi-step reasoning, tool use, memory, planning, and dynamic interaction patterns—and the corresponding challenges in deploying these systems reliably in production. Traditional serving frameworks often assume static inference graphs, making them ill-suited for agentic workloads that require flexible orchestration and adaptive execution.

The speakers then walk through how Apple leverages Ray as an Agent Engine, combining Ray Serve’s distributed execution model with its built-in autoscaling, request routing, and traffic management capabilities. This approach enables:

Framework-agnostic agent deployment, allowing teams to integrate agents built using any architecture or library

Dynamic, scalable execution for agents that require multi-step reasoning and complex control flows

High robustness and resiliency, even under unpredictable load patterns

Simplified operational workflows, reducing the burden of managing complex agent pipelines

They share practical lessons learned from deploying agentic systems internally at Apple, highlighting patterns that generalize to a wide variety of agent frameworks and application domains.

Attendees will leave with actionable insights on how to build scalable, resilient, and production-ready agent applications using Ray Serve—regardless of the underlying agent design or development workflow.


Liked this video? Check out other Ray Summit breakout session recordings youtube.com/playlist?list=PLzTswPQNepXllnU0C36WtkC0dqkAoDulh

Subscribe to our YouTube channel to stay up-to-date on the future of AI! youtube.com/c/anyscale

🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Ray Agent Engine: Deploying AI Agents with Ray Serve | Ray Summit 2025Planetary-scale multimodal image processing | Xoople + AnyscalePlatform Orchestration: Anyscale + Ray on Kubernetes | Anyscale | Ray Summit 2026How DataRobot Parallelizes Agentic Pipeline Searches with Ray + syftr | Ray Summit 2025Diffusion Large Language Models | Inception Labs | Ray Summit 2026Efficient, High-Performance AI Inferencing with Intel Xeon 6 | Ray Summit 2025Cascade: Write-Once Run-Anywhere Inference on Ray | Samsara | Ray Summit 2026Maximizing GB200 and GB300 Performance with Domain-Aware Scheduling | NVIDIA | Ray Summit 2026Ray Data on GPU - Breaking Preprocessing Bottleneck for Model Training | Stripe | Ray Summit 2026Miles: Enterprise-Facing Agentic RL Framework | RadixArk + Anyscale | Ray Summit 2026Beyond Text-to-Video: Omni-Context Generation in a 33B Model | MiniMax | Ray Summit 2026Inside NVIDIA Dynamo: Faster, Scalable AI Deployment | Ray Summit 2025
Anyscale |

Ray Agent Engine: Deploying AI Agents with Ray Serve | Ray Summit 2025

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER