Uploaded September 2026 | Updated November 2026, 2 hours ago
vLLM and AMD AITER are moving ROCm support beyond simple porting through workload-aware kernel orchestration, with benchmarks showing 1.2 to 4.4x throughput gains across AMD Instinct GPUs.
At Ray Summit 2026, Douglas Lehr, Engineering Fellow at AMD, covers ROCM_AITER_FA's explicit routing of mixed prefill, extend, and decode traffic to specialized kernels, along with batch reordering, chunked long-context processing, hardware-optimized KV cache layouts, and AITER MLA backends for DeepSeek-class models.
You'll leave with a clear picture of how software-hardware co-design turns ROCm into a high-performance production serving platform.
This session was part of the first vLLM Conference at Ray Summit, hosted by Inferact.
Liked this video? Check out other Ray Summit vLLM session recordings: youtube.com/playlist?list=PLXMguE8Nc9o4
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
vLLM and AMD AITER are moving ROCm support beyond simple porting through workload-aware kernel orchestration, with benchmarks showing 1.2 to 4.4x throughput gains across AMD Instinct GPUs.
At Ray Summit 2026, Douglas Lehr, Engineering Fellow at AMD, covers ROCM_AITER_FA's explicit routing of mixed prefill, extend, and decode traffic to specialized kernels, along with batch reordering, chunked long-context processing, hardware-optimized KV cache layouts, and AITER MLA backends for DeepSeek-class models.
You'll leave with a clear picture of how software-hardware co-design turns ROCm into a high-performance production serving platform.
This session was part of the first vLLM Conference at Ray Summit, hosted by Inferact.
Liked this video? Check out other Ray Summit vLLM session recordings: youtube.com/playlist?list=PLXMguE8Nc9o4
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










