vLLM's speculative decoding runs open models 2-3x faster on AMD
Draft-and-verify hits up to 2.87x on MI300X and MI355X across Gemma, Qwen and Kimi — but the speedup depends on tuning draft length per workload.
Copy markdown
The numbers: 2-3x on Instinct, no NVIDIA required
vLLM's benchmarks show speculative decoding running on AMD MI300X and MI355X: up to 2.87x on Gemma 4 26B, 2.68x on Kimi K2.5, and 2.20x on Qwen 3.5-122B. If H100s are out of reach or out of stock, that's real serving throughput on hardware you can actually get.
Five drafters, and they don't win the same races
The post pits native MTP, Gemma-4 MTP, EAGLE-3, DFlash and DSpark against each other. DFlash led on reasoning tasks like MATH500 and GSM8K, while sequential methods held up better on chat and general use — pick your drafter by workload, not by a single leaderboard.
The catch: acceptance falls off a cliff
Later draft tokens get rejected fast — acceptance can fall from 86% at position 1 to 4% by position 15, so proposing too many tokens actively slows you down. The best draft length swung from N=4-5 on code to N=7-11 on reasoning, and some configs showed no gain at all. Start at the checkpoint's recommended N and measure on your own traffic.
Elsewhere: Copilot adds GPT-6 Astra and Gemini 3.8 Flash
GitHub made GPT-6 Astra and Gemini 3.8 Flash generally available in Copilot and flagged a set of older models for retirement. Check your pinned model before it gets deprecated out from under you.