Serving vLLM on Agentic Production Workloads | Inferact | Ray Summit 2026 @anyscale
Serving vLLM on Agentic Production Workloads | Inferact | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 2 days ago
Agentic production traffic brings long multi-turn sessions, heavy prefix reuse, and bursty, heterogeneous requests that challenge conventional serving.

At Ray Summit 2026, Yifan Qiao and Zijing Liu from Inferact explain how vLLM addresses those patterns through independently scaled prefill/decode disaggregation, turn-aware serving, and session and correlation hints that improve prefix caching, prefetching, and eviction. They also cover distributed KV cache transfer and storage with Mooncake and multi-tier offloading, allowing cached context to move across turns and instances instead of being recomputed.

You'll leave with practical guidance for architecting and tuning vLLM deployments for production agents.

This session was part of the first vLLM Conference at Ray Summit, hosted by Inferact.

Liked this video? Check out other Ray Summit vLLM session recordings: youtube.com/playlist?list=PLXMguE8Nc9o4

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Serving vLLM on Agentic Production Workloads | Inferact | Ray Summit 2026Cost-Effective RL Post-Training with Ray and AWS Trainium | AWS + Anyscale | Ray Summit 2026Ray Serve: Advancing scalability and flexibility | Ray Summit 2025Trillion-Token Pretraining | Adyen | Ray Summit 2026Running Ray in Production: Google’s Guide to Operators & Observability | Ray Summit 2025From Agents to Action: Scaling Agentic AI with Ray on Intel Xeon | Intel | Ray Summit 2026Building an AI Science Factory | Andrew Beam (Lila Sciences) | Ray Summit 2026GLM-5 RL Training with vLLM | Prime Intellect | Ray Summit 2026Accelerating vLLM with LMCache | Ray Summit 2025Ray History Server: Observability for Ephemeral K8s Clusters | KubeRay | Ray Summit 2026SQL or Python? Why It’s the Wrong Question in the AI Era | Ray Summit 2025Brendan Burns: Lessons from Building Kubernetes and the Future of AI Infrastructure
Anyscale |

Serving vLLM on Agentic Production Workloads | Inferact | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER