Serving a Frontier Open Model with Ray & vLLM | Nscale | Ray Summit 2026 @anyscale
Serving a Frontier Open Model with Ray & vLLM | Nscale | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 2 days ago
A simpler setup delivered more tokens per second, with the first token arriving sooner for a single request, than the model card's reference configuration.

At Ray Summit 2026, Eyal Lantzman, VP of AI Services at Nscale, uses DeepSeek-V4-Flash, Ray, and vLLM to explore the practical tradeoffs between scaling strategies, including tensor and expert parallelism, prefill/decode disaggregation, and whole-replica scale-out, plus the production caveats that matter at scale, from GPU topology and cache pressure to long context and speculative decoding.

You'll leave with what worked, what didn't, and how to choose a scaling strategy for frontier open models.

Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Serving a Frontier Open Model with Ray & vLLM | Nscale | Ray Summit 2026Multi-Region, Environment Isolated Ray Environments in the Notion Way | Notion | Ray Summit 2026Pinterest’s Approach to Real-Time ML Experimentation Using Ray | Ray Summit 2025From OOMs to 2x Throughput: Scaling Multimodal Data Curation on Ray | Zoox | Ray Summit 2026Custom Pipeline Parallelism for Frontier-Scale Model Serving | Apple | Ray Summit 2026Scaling LinkedIns Online Training Solution with Ray | Ray Summit 2025AI workloads introduce a system design shift ...Inside Lilas AI Research Platform | Lila Sciences | Ray Summit 2026LLM Post-Training and High-Performance Serving | Anyscale | Ray Summit 2026Training Frontier Knowledge Work Agents: A 397B Open Recipe with SkyRL | Mercor | Ray Summit 2026Building Self-Serve, Multimodal Medical Image Pipelines with Ray | Heartflow | Ray Summit 2026Agents and Robots: The Human Effort Putting AI Into Production | Encord | Ray Summit 2026
Anyscale |

Serving a Frontier Open Model with Ray & vLLM | Nscale | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER