Uploaded October 2025 | Updated September 2026, 2 weeks ago

---

Thanks to Supabase for sponsoring this video, check them out: https://supabase.plug.dev/UGNnsbw

Learn why the same 1T open-weight model can perform widely differently, depending on the inference provider. We will look at a Kimi K2 vendor verifier benchmark that compares open router providers, highlight real benchmark gaps accuracy with cost slash latency and show what to check before blaming the model weights.


Website: engineerprompt.ai

RAG Beyond Basics Course:
prompt-s-site.thinkific.com/courses/rag

Let's Connect:
🦾 Discord: discord.com/invite/t4eYQRUcXB
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Patreon: patreon.com/PromptEngineering
💼Consulting: calendly.com/engineerprompt/consulting-call
📧 Business Contact: engineerprompt@gmail.com
Become Member: tinyurl.com/y5h28s6h

💻 Pre-configured localGPT VM: bit.ly/localGPT (use Code: PromptEngineering for 50% off).

Signup for Newsletter, localgpt:
https://tally.so/r/3y9bb0

TIMESTAMP:

00:00 The Problem with Open Rate Models
01:14 Not All APIs are the same
02:16 KIMI K2 - Vendor Verifier
04:42 Why we see variations
06:54 Understanding Tool Calls in Agent Systems
08:23 Backend as a Service: Supabase
09:52 How to think about benchmark design