Uploaded October 2026 | Updated November 2026, 1 day ago
I wanted my local AI to choose the right model automatically instead of making me pick between speed and capability for every request.
I test NVIDIA Switchyard for local AI model routing with Nemotron 3.5 Lightning on an RTX 3090 and DeepSeek V4 Flash across two GX10s. I cover Stage routing, vLLM, context windows, and some of the problems I ran into while testing it with real coding agents.
In my best autonomous coding run, almost 88% of the routing decisions went to Nemotron while DeepSeek was still available when Switchyard decided it was needed.
This is pretty close to what I wanted. Most of the work stays on the fast local model, the more capable model gets pulled in when it makes sense, and I don't have to choose between them before I start.
Video Notes: technotim.com/posts/switchyard-local-ai-routing
Merch Shop 🛍️: https://l.technotim.com/shop
Support me on Patreon: patreon.com/technotim
Sponsor me on GitHub: github.com/sponsors/timothystewart6
Subscribe on Twitch: twitch.tv/technotim
Become a YouTube member: youtube.com/channel/UCOk-gHyjcWZNj3Br4oxwh0A/join
Gear Recommendations: https://l.technotim.com/gear
Get Help in Our Discord Community: https://l.technotim.com/discord
2nd channel: youtube.com/@TechnoTimTinkers
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 Automatic AI Model Routing
01:09 Nemotron Lightning vs DeepSeek
01:40 Fixing Nemotron's Performance
02:35 Is Nemotron Actually Good?
02:58 How NVIDIA Switchyard Works
04:55 Putting Stage Routing to the Test
05:28 The 32K Context Problem
06:46 Fixing Routing with 128K Context
07:44 Testing on Real-World Code
08:40 Building the Autonomous Agent Test
09:46 Run 4: Getting Close
10:39 Run 5: 88% Routed to Nemotron
11:50 Run 6: When the Agent Got Stuck
12:11 Is NVIDIA Switchyard Worth It?
13:40 What's Next for My Local AI
Thank you for watching!
I wanted my local AI to choose the right model automatically instead of making me pick between speed and capability for every request.
I test NVIDIA Switchyard for local AI model routing with Nemotron 3.5 Lightning on an RTX 3090 and DeepSeek V4 Flash across two GX10s. I cover Stage routing, vLLM, context windows, and some of the problems I ran into while testing it with real coding agents.
In my best autonomous coding run, almost 88% of the routing decisions went to Nemotron while DeepSeek was still available when Switchyard decided it was needed.
This is pretty close to what I wanted. Most of the work stays on the fast local model, the more capable model gets pulled in when it makes sense, and I don't have to choose between them before I start.
Video Notes: technotim.com/posts/switchyard-local-ai-routing
Merch Shop 🛍️: https://l.technotim.com/shop
Support me on Patreon: patreon.com/technotim
Sponsor me on GitHub: github.com/sponsors/timothystewart6
Subscribe on Twitch: twitch.tv/technotim
Become a YouTube member: youtube.com/channel/UCOk-gHyjcWZNj3Br4oxwh0A/join
Gear Recommendations: https://l.technotim.com/gear
Get Help in Our Discord Community: https://l.technotim.com/discord
2nd channel: youtube.com/@TechnoTimTinkers
(Affiliate links may be included in this description. I may receive a small commission at no cost to you.)
00:00 Automatic AI Model Routing
01:09 Nemotron Lightning vs DeepSeek
01:40 Fixing Nemotron's Performance
02:35 Is Nemotron Actually Good?
02:58 How NVIDIA Switchyard Works
04:55 Putting Stage Routing to the Test
05:28 The 32K Context Problem
06:46 Fixing Routing with 128K Context
07:44 Testing on Real-World Code
08:40 Building the Autonomous Agent Test
09:46 Run 4: Getting Close
10:39 Run 5: 88% Routed to Nemotron
11:50 Run 6: When the Agent Got Stuck
12:11 Is NVIDIA Switchyard Worth It?
13:40 What's Next for My Local AI
Thank you for watching!










