NVIDIA open-sources NeMo Switchyard, a self-hostable agent router
An Apache-2.0 proxy that sends each agent step to the cheapest capable model — up to 74% off in outside tests. Plus the Nemotron it hands the cheap work to.
Copy markdown
It routes each step, not the whole job
Switchyard sits in front of your LLM calls and picks the cheapest model that can handle each request in an agent loop, saving frontier models for the calls that actually need them. It's an open-source (Apache 2.0) Python proxy whose reference server speaks OpenAI, Anthropic, and Responses API formats, so it drops in without a rewrite.
The savings, and the catch
On LangChain's public eval it cut cost ~74% versus a frontier-only baseline while sending just 7% of calls to the expensive model — at a roughly 6-point accuracy hit. Cognition reported 28% lower cost within 2.8 points of frontier. How much you save hinges on how forgiving your task is.
Four routers, zero training
Three work out of the box: an LLM-as-judge classifier, a stage router that adapts to where the agent is, and an escalation router that starts cheap and climbs only when a model fails. A tunable prefill router reads model internals to predict success if you want to push further.
The cheap end: Nemotron 3.5 Lightning
NVIDIA shipped Switchyard alongside Nemotron 3.5 Lightning, a small, fast model aimed at the long-running specialized steps the router hands off — the low-cost tier that makes the routing math work in the first place.
Try it this weekend
Because it's a drop-in proxy, you can point an existing OpenAI or Anthropic client's base URL at a local Switchyard server and watch it shuffle models mid-task — a concrete way to benchmark your own agent's cost/quality curve before committing to it.