Uploaded October 2026 | Updated November 2026, 1 day ago
Most teams spin up a new open model with the default vLLM recipe and hope for the best. Runpod forward-deployed engineer Chris Vela shows how to choose configs from data instead, starting with the use case and concurrency, and demos BenchPod, a benchmarking dashboard built on vLLM that tests configs live without restarts. He runs Kimi K3 on a B300, shows how tuning max batch tokens cuts inter-token latency, and asks whether you need a frontier model at all.
Chapters:
0:00 Intro
0:29 What Runpod does
1:32 Everyone's guessing at vLLM configs
2:53 Version and CUDA gotchas
3:48 BenchPod: benchmarking on top of vLLM
6:07 Running Kimi K3 on a B300
7:40 Demo
10:01 Tuning max batch tokens
13:34 From R&D to production
14:21 Do you need the frontier model?
Chris Vela
runpod.io
Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!
Most teams spin up a new open model with the default vLLM recipe and hope for the best. Runpod forward-deployed engineer Chris Vela shows how to choose configs from data instead, starting with the use case and concurrency, and demos BenchPod, a benchmarking dashboard built on vLLM that tests configs live without restarts. He runs Kimi K3 on a B300, shows how tuning max batch tokens cuts inter-token latency, and asks whether you need a frontier model at all.
Chapters:
0:00 Intro
0:29 What Runpod does
1:32 Everyone's guessing at vLLM configs
2:53 Version and CUDA gotchas
3:48 BenchPod: benchmarking on top of vLLM
6:07 Running Kimi K3 on a B300
7:40 Demo
10:01 Tuning max batch tokens
13:34 From R&D to production
14:21 Do you need the frontier model?
Chris Vela
runpod.io
Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!










