Uploaded September 2026 | Updated November 2026, 3 days ago
DigitalOcean's vLLM deployments reached leading Artificial Analysis results across frontier open-weight models, including 230 tokens per second per user on DeepSeek V3.2 and sub-second time to first token for 10,000-token Qwen 3.5 prompts.
At Ray Summit 2026, Debarshi Raha, VP and Fellow Engineer at DigitalOcean, breaks down the open-source optimization work behind those results, from low-batch kernel fusion and EAGLE3 speculative decoding to linear-attention fusions and dual-stream execution.
You'll leave seeing how profiling model-specific bottlenecks and upstreaming the fixes into vLLM delivers production inference performance that rivals proprietary stacks.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
DigitalOcean's vLLM deployments reached leading Artificial Analysis results across frontier open-weight models, including 230 tokens per second per user on DeepSeek V3.2 and sub-second time to first token for 10,000-token Qwen 3.5 prompts.
At Ray Summit 2026, Debarshi Raha, VP and Fellow Engineer at DigitalOcean, breaks down the open-source optimization work behind those results, from low-batch kernel fusion and EAGLE3 speculative decoding to linear-attention fusions and dual-stream execution.
You'll leave seeing how profiling model-specific bottlenecks and upstreaming the fixes into vLLM delivers production inference performance that rivals proprietary stacks.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










