Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM @infoq
Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM  @infoq
Uploaded November 2026 | Updated November 2026, 1 day ago
In this episode, Edoardo Vacchi discusses the evolving landscape of WebAssembly on the backend, running Java workloads on GPUs using TornadoVM, and optimizing distributed AI inference using llm-d and vLLM on Kubernetes. Edoardo explains how llm-d manages cache locality (KV and prefix caches), request routing, and batch inference across Kubernetes clusters to maximize expensive GPU utilization.

Read a transcript of this interview: bit.ly/4eLRHnE

Newsletter:
Subscribe to the Software Architects' Newsletter, a monthly roundup of the patterns and technologies senior practitioners are working through, with the news and lessons from people doing the work: infoq.com/software-architects-newsletter

InfoQ Online Certification Programs:
InfoQ’s online cohorts put you in a small group of senior engineers and architects from other companies who face the same decisions. You bring a real decision from your job and work through it using approaches from QCon talks. You write up what you learned as an article, which makes you put the trade-offs into words before your next proposal at work. You leave with new approaches, or confirmation that the calls you’re already making are the right ones. See dates:
certification.qconferences.com


Upcoming Events:
QCon San Francisco 2026 (November 16-20, 2026)
qconsf.com

QCon AI New York 2026 - AI Engineering Conference (December 15-16, 2026)
newyork.qcon.ai

QCon London 2027 (April 13-16, 2027)
qconlondon.com

QCon AI Boston 2027 - AI Engineering Conference (July 8-9, 2027)
boston.qcon.ai


The InfoQ Podcasts:
Weekly conversations with senior software leaders about how they build systems and teams, including what they'd do differently. Listen to all our podcasts and read interview transcripts:
The InfoQ Podcast: infoq.com/podcasts
Engineering Culture Podcast by InfoQ: infoq.com/podcasts/#engineering_culture
Generally AI: infoq.com/generally-ai-podcast
Follow InfoQ:
Mastodon: https://techhub.social/@infoq
X: https://x.com/InfoQ
LinkedIn: linkedin.com/company/infoq
Facebook: facebook.com/InfoQdotcom
Instagram: instagram.com/infoqdotcom
YouTube: youtube.com/infoq
Bluesky: https://bsky.app/profile/infoq.com

Write for InfoQ:
Share what you've learned building software with a community of senior practitioners, and get your work in front of the people who read InfoQ. infoq.com/write-for-infoq
Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVMHow SBOMs and Engineering Discipline Can Help You Avoid Trivy’s CompromiseThe Future of AI: From Enterprise Adoption to Open Source SovereigntyHow Nubank Scales: From Binary Hell to Server-Driven UIBeyond the Hype: 28x Your Engineering Velocity with AIData Centers Use HOW Much Power?!  #infoqA Java Performance Quest: Taming Unsafe Code, Embracing Idiomatic Style & Debugging the Linux KernelIs AI Killing the Next Generation of Engineers?90% of Devs Get Java Serverless WRONG (How to Fix It)The real skill in 2026? Orchestrating an army of #AIagents.Running Payments on AWS, GCP and Azure at OnceRoblox Engineering Director: Prompt to Prod Is Broken Without Trust
InfoQ |

Distributed LLM Inference on Kubernetes with llm-d, Wasm, & TornadoVM

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER