Uploaded September 2026 | Updated November 2026, 4 days ago
As agentic AI drives ever-longer context windows, KV cache has become the new memory wall, dwarfing GPU HBM capacity and forcing costly recomputation on every session resume.
At Ray Summit 2026, Vaughn Stewart, VP of Systems Engineering at VAST Data, covers how offloading KV cache to VAST's flash-based storage with NVIDIA Dynamo and BlueField-4/CMX architecture eliminates that tax: 20x faster time to first token, 90 percent GPU time savings, and up to 64 percent lower three-year TCO.
You'll leave with sizing guidance, benchmark data, and a blueprint for scaling inference context without scaling GPU spend.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
As agentic AI drives ever-longer context windows, KV cache has become the new memory wall, dwarfing GPU HBM capacity and forcing costly recomputation on every session resume.
At Ray Summit 2026, Vaughn Stewart, VP of Systems Engineering at VAST Data, covers how offloading KV cache to VAST's flash-based storage with NVIDIA Dynamo and BlueField-4/CMX architecture eliminates that tax: 20x faster time to first token, 90 percent GPU time savings, and up to 64 percent lower three-year TCO.
You'll leave with sizing guidance, benchmark data, and a blueprint for scaling inference context without scaling GPU spend.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
🔗 Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










