llama.cpp v0.9.0 adds multi-node tensor parallelism and FP8
A new NCCL backend shards one model across several machines, OCP FP8 E4M3 quantization arrives, and Mistral Large 4 and Chatterbox TTS get day-one support.

Copy markdown
Shard one model across several machines
The headline is NCCL-based multi-node tensor parallelism: llama.cpp can now split a single model across GPUs on different machines, not just one box. That's the path to running weights too big for any one rig — pool a couple of workstations instead of renting a datacenter card.
Native OCP FP8 E4M3 quantization
A new FP8 E4M3 tensor type (saturating mode) lands with a LLAMA_FTYPE_MOSTLY_F8_E4M3 ftype and conversion helpers, matching the 8-bit format modern GPUs run natively. It ships alongside a flash_attn_ext_rows op that restores fast prefill for multi-sequence, unified-KV serving.
Mistral Large 4, Chatterbox TTS, offline transcription
Four new architectures get support — Mistral Large 4, Clover, Limite and Metis-1.6 — so you can run Mistral's latest in GGUF locally. Audio expands too: Chatterbox TTS, Pocket TTS preset voices, and Nemotron 3.5 Transcribe for offline speech-to-text in 40 languages with speaker attribution. Gemma 4 E2B/E4B also pick up big prefill speedups.