Accelerating Large Language Models: vLLM on TPUs | Google | Ray Summit 2026 @anyscale
Accelerating Large Language Models: vLLM on TPUs | Google | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 3 days ago
Large language models demand significant computational power for inference, and TPUs are becoming a first-class target for serving them.

At Ray Summit 2026, Qi Zhou, Senior Staff Software Engineer at Google, explores the work to leverage Google TPUs with vLLM, orchestrated by Ray. He covers the performance and cost-efficiency benefits of TPU-based LLM serving, key enablers like the TorchTPU backend that brings a native PyTorch experience to TPUs, and insights from optimizing and scaling vLLM deployments.

You'll leave with the architectural approaches, the challenges overcome, and the outlook for high-performance, TPU-accelerated LLM inference with vLLM and Ray.

This session was part of the first vLLM Conference at Ray Summit, hosted by Inferact.

Liked this video? Check out other Ray Summit vLLM session recordings: youtube.com/playlist?list=PLXMguE8Nc9o4

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Accelerating Large Language Models: vLLM on TPUs | Google | Ray Summit 2026KubeSecRay: Fortifying Multi-Tenant Ray Clusters on Kubernetes | Ray Summit 2024Scaling the Ray Serve Controller to Thousands of Replicas | Anyscale | Ray Summit 2026Taming Distributed AI Training with Ray + Datadog Observability | Ray Summit 2025Webinar: Scaling LLM Fine-Tuning with FSDP, DeepSpeed, and RayFrom Prototype to Production: Securely Accelerating Physical AI with Vision-Language-Action ModelsClosing the Train-Inference Divide for Asynchronous RL at Scale | Lila Sciences | Ray Summit 2026SkyRL tx: A unified training and inference engine | Ray Summit 2025Scaling Production LLM Inference Using EKS Auto Mode & Ray Serve | Ray Summit 2025From Self-Managed Ray to Anyscale Services: Migrating ML Serving | Coinbase | Ray Summit 2026Where’s My Training Job? Ray & GPU Observability at Datadog | Datadog | Ray Summit 2026Serving a Frontier Open Model with Ray & vLLM | Nscale | Ray Summit 2026
Anyscale |

Accelerating Large Language Models: vLLM on TPUs | Google | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER