AMD and vLLM: Whats New | AMD | Ray Summit 2026 @anyscale
AMD and vLLM: Whats New | AMD | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 2 hours ago
vLLM and AMD AITER are moving ROCm support beyond simple porting through workload-aware kernel orchestration, with benchmarks showing 1.2 to 4.4x throughput gains across AMD Instinct GPUs.

At Ray Summit 2026, Douglas Lehr, Engineering Fellow at AMD, covers ROCM_AITER_FA's explicit routing of mixed prefill, extend, and decode traffic to specialized kernels, along with batch reordering, chunked long-context processing, hardware-optimized KV cache layouts, and AITER MLA backends for DeepSeek-class models.

You'll leave with a clear picture of how software-hardware co-design turns ROCm into a high-performance production serving platform.

This session was part of the first vLLM Conference at Ray Summit, hosted by Inferact.

Liked this video? Check out other Ray Summit vLLM session recordings: youtube.com/playlist?list=PLXMguE8Nc9o4

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
AMD and vLLM: Whats New | AMD | Ray Summit 2026How Zoox Built a Reliable, High-Velocity Model Serving Platform with Ray Serve | Ray Summit 2025Scaling Machine Learning at Tripadvisor: Our Journey with Ray and Anyscale | Ray Summit 2025Lightning on Agentic: Embedded LLM, Anyscale, Novita, and Verda | Ray Summit 2026Async Inference at Netflix: Batch and Near-Real-Time Serving | Netflix | Ray Summit 2026AWS + vLLM: Building the Future of Open, Fast LLM Serving | Ray Summit 2025Transforming Grab’s AI Platform with Ray | Grab | Ray Summit 2026Ray + vLLM  Efficient Multi Node Orchestration for Sparse MoE Model Serving | Ray Summit 2025Hybrid RL + Imitation Learning for Robotics with Ray at RAI InstituteVideo Action Models | mimic robotics | Ray Summit 2026How Runhouse Orchestrates Multi-Cluster Ray Workloads | Ray Summit 2025Scaling DSpark Training Using vLLM, Speculators and Mooncake | Red Hat | Ray Summit 2026
Anyscale |

AMD and vLLM: What's New | AMD | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER