NVIDIA's open VoiceChat 11B: full-duplex voice agents with tool calls
One 11B network replaces the ASR-LLM-TTS cascade with 448ms turn-taking and barge-in; the catch is one 80GB GPU, research-only terms, and English only.
Copy markdown
One model, not three services
VoiceChat fuses a Fast Conformer encoder, the Nemotron Nano v2 LLM, and an NVIDIA TTS decoder into a single full-duplex network, retiring the Whisper-plus-LLM-plus-TTS stack most voice apps still stitch together. Measured turn-taking is 448ms, and it handles barge-in when you cut in mid-sentence.
The first open voice model that calls tools mid-sentence
It emits <TOOLCALL> blocks on a side channel and plays operator-defined 'on-hold' lines while your API runs, so the agent can look something up or transact without going silent. It averages 56.1% on the BFCL-v3 tool-calling harness.
Read the catch before you plan a product
The OpenMDW-1.1 weights are labeled 'research purposes only,' need one 80GB GPU (A100, H100, RTX 6000 Pro, or B200), and speak English only. Known failure modes: a two-minute audio ceiling, gibberish after several turns, and runaway self-talk after a reply ends.
Build this weekend: a local voice agent
NVIDIA shipped a Nemotron Voice Agent Blueprint and Daily.co published a wiring guide, so you can drop VoiceChat into a real-time pipeline today. Would rather rent than run 80GB? Yesterday's Beijing-only Qwen Realtime API is the hosted counterpart.
Elsewhere: OpenRouter routes by 'wisdom of the market'
OpenRouter's revamped Auto router now picks models from the aggregate choices of millions of users and claims it outperforms task-based classifiers across the board, so one endpoint replaces your hand-tuned routing rules.