Liquid AI 4-bit checkpoints keep 97% of quality for on-device agents
Quantization-aware distillation recovers what plain 4-bit throws away — 230M to 2.6B models, GGUF-ready, and decoding faster than Q5 on a phone or a Pi 5.
Copy markdown
97% of full precision, at four bits
Liquid AI shipped Q4_0 checkpoints for LFM2.5 trained with quantization-aware distillation — a BF16 teacher distilled into the quantized student — holding 96.5–97.4% of full-precision scores across GPQA Diamond, MMLU-Pro and IFEval, where naive round-to-nearest 4-bit usually sheds far more. Sizes ship at 230M, 350M, 1.2B-Instruct and 2.6B.
Smaller and faster than a Q5 build
On a Galaxy S26 Ultra, a Raspberry Pi 5, a MacBook Pro and a mini-PC, the 230M and 350M checkpoints decode 4–33% faster than a Q5_K_M build at matched quality — so you shed memory and gain throughput at once, instead of trading one for the other.
Pull one GGUF and run it tonight
All four checkpoints are live as GGUF on Hugging Face; the one-liner `llama-cli -hf LiquidAI/LFM2.5-350M --hf-file LFM2.5-350M-QAD-Q4_0.gguf` gets you a tool-calling on-device model with no cloud key, no GPU, running fully local.