Ask the Experts: How NeMo Switchyard Helps Agents Select Models  | Nemotron Labs @NVIDIADeveloper
Ask the Experts: How NeMo Switchyard Helps Agents Select Models  | Nemotron Labs  @NVIDIADeveloper
Uploaded September 2026 | Updated November 2026, 5 days ago
NVIDIA NeMo Switchyard routes AI agents to the right model at the right time — cutting token costs by up to 80% while maintaining frontier accuracy.

In this Nemotron Labs Ask the Experts session, NVIDIA engineers Nave Algarici and Ryan Lempka explain how NeMo Switchyard works, how it differs from a simple model router, and what's coming on the roadmap. Switchyard operates on a provider-neutral format across all routing algorithms, stores state across agent trajectories, and makes per-turn routing decisions based on tool call history, response quality, and agent performance signals — not just the query.

Topics covered: why systems of models outperform single-model setups, how to integrate Switchyard directly into an agent harness or an LLM gateway like OpenRouter or LiteLLM, the Switchyard roadmap (agent-aware routing, continual learning via NeurRelay, heterogeneous GPU scaling with NVIDIA Dynamo, privacy routing with Nemo Anonymizer), observability via OpenTelemetry, KV cache cost modeling, edge device deployment, and how Switchyard maintains agent authority boundaries during dynamic routing.

Speakers:
Nave Algarici, Product Management Lead, NVIDIA NeMo
Ryan Lempka, Core Engineer, NeMo Switchyard, NVIDIA
Host: Chris Alexiuk

Resources:
NeMo Switchyard Github → github.com/NVIDIA-NeMo/Switchyard

CHAPTERS:
0:00 Intro & speaker introductions
3:00 Why systems of models — and why now
6:00 What is NeMo Switchyard and why was it built
10:00 How Switchyard differs from a simple router
13:00 Switchyard roadmap (agent-aware routing, NeurRelay, Dynamo, privacy routing, flywheel)
18:00 Can I stream Grafana-style metrics from Switchyard?
19:00 Does Switchyard show xAI-style visibility into routing choices?
22:00 When to use direct-to-agent vs. gateway configuration
25:00 How do you prevent agent authority from expanding during dynamic routing?
28:00 Does Switchyard work on edge devices like Jetson?
30:00 Managing token budgets for agents and humans
33:00 Does Switchyard take advantage of cross-model KV cache transfer?
37:00 How does Switchyard learn from agent traces?
40:00 What problem does NeMo Switchyard solve? (revisited)
43:00 Resources
46:00 How did Nave and Ryan get into model routing at NVIDIA?

#nvidia #nemotron #modelrouting
#aiagents #llm #agenticai #machinelearning #opensource #developertools #localai
Ask the Experts: How NeMo Switchyard Helps Agents Select Models  | Nemotron LabsFine-Tuning Financial LLMs for Customer Intent and ExperienceDGX Spark: Observing the Universe at Light SpeedYocto Project on Jetson: From Community Project to Official NVIDIA SupportRoad to GTC Berlin: Europe’s AI Builders in the SpotlightGenerating Autonomous Vehicles Neural Reconstruction With Open-Source Physical AI Agent SkillsNemotron 3.5 Lightning Q&A: Multimodal, RTX 6000What is an omni-model?From Video to Voice: Build Faster with TensorRT Model ConnectDGX Spark Live: Autonomous AI Agent MigrationIntroducing Nemotron 3 UltraGeometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior
NVIDIA Developer |

Ask the Experts: How NeMo Switchyard Helps Agents Select Models | Nemotron Labs

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER