I Made My Local AI Pick Its Own Model @TechnoTim
I Made My Local AI Pick Its Own Model  @TechnoTim
Uploaded October 2026 | Updated November 2026, 1 day ago
I wanted my local AI to choose the right model automatically instead of making me pick between speed and capability for every request.

I test NVIDIA Switchyard for local AI model routing with Nemotron 3.5 Lightning on an RTX 3090 and DeepSeek V4 Flash across two GX10s. I cover Stage routing, vLLM, context windows, and some of the problems I ran into while testing it with real coding agents.

In my best autonomous coding run, almost 88% of the routing decisions went to Nemotron while DeepSeek was still available when Switchyard decided it was needed.

This is pretty close to what I wanted. Most of the work stays on the fast local model, the more capable model gets pulled in when it makes sense, and I don't have to choose between them before I start.

Video Notes: technotim.com/posts/switchyard-local-ai-routing

Merch Shop 🛍️: https://l.technotim.com/shop
Support me on Patreon: patreon.com/technotim
Sponsor me on GitHub: github.com/sponsors/timothystewart6
Subscribe on Twitch: twitch.tv/technotim
Become a YouTube member: youtube.com/channel/UCOk-gHyjcWZNj3Br4oxwh0A/join
Gear Recommendations: https://l.technotim.com/gear
Get Help in Our Discord Community: https://l.technotim.com/discord
2nd channel: youtube.com/@TechnoTimTinkers

(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)

00:00 Automatic AI Model Routing
01:09 Nemotron Lightning vs DeepSeek
01:40 Fixing Nemotron's Performance
02:35 Is Nemotron Actually Good?
02:58 How NVIDIA Switchyard Works
04:55 Putting Stage Routing to the Test
05:28 The 32K Context Problem
06:46 Fixing Routing with 128K Context
07:44 Testing on Real-World Code
08:40 Building the Autonomous Agent Test
09:46 Run 4: Getting Close
10:39 Run 5: 88% Routed to Nemotron
11:50 Run 6: When the Agent Got Stuck
12:11 Is NVIDIA Switchyard Worth It?
13:40 What's Next for My Local AI

Thank you for watching!
I Made My Local AI Pick Its Own ModelHomeLab Hardware Tour! (Late 2023)Meet Uptime Kuma, a Fancy Open Source Uptime Monitor for all your HomeLab Monitoring NeedsHuge Network Upgrade for My HomelabPerfect Proxmox Template with Cloud Image and Cloud InitMy HomeLab Tour #shorts #homelab #proxmox #truenasA Hypervisor Built on Kubernetes - Cloud Native HCI with HarvesterIll NEVER run out of POWER! - Eaton and Tripp Lite UPSTechno Tim HomeLab and NEW Server Room Tour!  (Late 2021)2 Factor Auth and Single Sign On with AutheliaStatic site generators are awesome #shortsI Built My Own Intelligent AI Gateway
Techno Tim |

I Made My Local AI Pick Its Own Model

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER