Why round-robin load balancing destroys your KV cache @redhat
Why round-robin load balancing destroys your KV cache  @redhat
Uploaded September 2026 | Updated November 2026, 3 days ago
Greg Pereira, Senior Machine Learning Engineer at Red Hat, discusses the scale-out problem: add a second vLLM replica behind a traditional load balancer and you drop to roughly 50% KV cache reuse. It only gets worse from there.

From vLLM Office Hours #55: Mooncake + vLLM/llm-d Deep Dive. Full session: youtu.be/RFyeBEy1AP8

Join us live every other Thursday: https://red.ht/office-hours

#vLLM #llm-d #KVCache #AIInference #RedHat #MLOps
Why round-robin load balancing destroys your KV cacheWhy digital innovation needs a global commonsLessons from the field with Red Hat OpenShiftRandom Acts Of AutomationGitOps Guide to the Galaxy (ep 112) | Farewell and Following Seas HilliaryWhy dedicated AI platforms failGitOps Guide to the Galaxy (ep 107) | Intro to Project VailLayered Zero Trust | Ask an OpenShift Expert 185Matt Quill, F5: Scaling For Complexity With Container AdoptionWhats new and whats next for Red Hat AI | Q1 2026Automate network security for AIWhat makes OpenShift a complete application platform?
Red Hat |

Why round-robin load balancing destroys your KV cache

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER