Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod @aiDotEngineer
Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod  @aiDotEngineer
Uploaded October 2026 | Updated November 2026, 1 day ago
Most teams spin up a new open model with the default vLLM recipe and hope for the best. Runpod forward-deployed engineer Chris Vela shows how to choose configs from data instead, starting with the use case and concurrency, and demos BenchPod, a benchmarking dashboard built on vLLM that tests configs live without restarts. He runs Kimi K3 on a B300, shows how tuning max batch tokens cuts inter-token latency, and asks whether you need a frontier model at all.

Chapters:
0:00 Intro
0:29 What Runpod does
1:32 Everyone's guessing at vLLM configs
2:53 Version and CUDA gotchas
3:48 BenchPod: benchmarking on top of vLLM
6:07 Running Kimi K3 on a B300
7:40 Demo
10:01 Tuning max batch tokens
13:34 From R&D to production
14:21 Do you need the frontier model?

Chris Vela
runpod.io

Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, RunpodClaude Managed Agents Q&A: Evals, CI/CD and Multi-Tenancy (Part 4 of 4) — AnthropicAI Cant Make Up a Number It Cant Look Up - Vinoo Ganesh, KeplerCan AI Agents Beat Wall Street at Predicting Earnings? - Ashwin Paranjape, Samaya AIHow to Know If Your AI Agent Is Any Good: Evals With Phoenix & Claude — Laurie Voss, Arize AIJudge a Transaction by the Company It Keeps — Somya Rai & MD Sharique, EXLYour AI Agent Keeps Making the Same Mistakes - Sparsh Jain, CentralAgent AIYour GPU Neighbor Can Hear Your Model — Branislav Brzak, StealthiumAI Labs Will Trap You or Replace You - Dillon Rolnick, Nous ResearchWhy Agentic Systems Need Ontologies — Frank Coyle, UC BerkeleyOur Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, RampThe AI Bill Is Too Damn High: Tokenomics 101 — Eugene Cheah, Featherless AI
AI Engineer |

Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER