Ornith-1.5 ships open DFlash draft models for ~1.9x faster serving

Apache-2.0 draft models drop into SGLang and verify every token, so you get 40-90% more throughput on one GPU with output identical to greedy decoding.

Nowline OCT 1 12:00 PM banner

Top AI stories from the last hour

Top AI stories from the last hour

Copy markdown

  • Free throughput, verified token-for-token

    A 0.5B draft model accelerates Ornith-1.5-35B-A3B to ~473 tok/s on an H200 versus 254 without it (~1.9x), and 44% faster on an NVIDIA DGX Spark, at concurrency 1. The target verifies every proposed token, so output is identical to greedy decoding — roughly 4 tokens land per step.

  • Turn it on in SGLang today

    Weights are on Hugging Face now under Apache-2.0 (the Ornith-1.5 target is MIT, built on Qwen3.5). Serve it with SGLang's DFlash2 support: `--speculative-algorithm DFLASH --speculative-num-draft-tokens 10`. A quantized NInfer C++ build packs the same draft head into ~21GB for a single 24GB card.

  • DFlash, the trick behind the speedup

    DFlash drafts a whole block of tokens in parallel with a block-diffusion model and injects the target's KV features into every draft layer. On Qwen 3.5 397B it topped 4.3x throughput on HumanEval (1.5x over native MTP); on small models it beats EAGLE-3, 3.2x vs 2.2x. It rides SGLang's Spec V2 engine, so the win generalizes beyond Ornith.