2.3% of Ollama's official quants are broken — a free tool finds them
A registry census found low-bit quants of small models broken for months, with no warning on the page. Perplexity flags them; a free tool runs the screen.
Copy markdown
Broken quants have been live on Ollama for months
A census of 327 code-capable GGUFs found 7 of 305 official Ollama library tags (2.3%) that solved at most 6 of 164 HumanEval+ coding tasks — effectively dead — and five of them had been served unchanged from August to October with nothing on the page to warn you. Every confirmed failure was a low-bit quant of a model at or below 4B params, so your small local coders are exactly where the risk concentrates.
Perplexity 27–5,700x over fp16 is the tripwire
The broken builds measured 27 to 5,700 times the perplexity of their fp16 source, so a quick perplexity pre-check against a reference catches most of them before they ship. But a file can match its build recipe exactly and still be broken — only execution testing (a short code screen, then a full rerun on the suspects) is a real gate, not metadata or a matching checksum.
quantcheck runs the screen; pin by digest, not tag
The authors released quantcheck — the screen, scorer, and rebuild/tensor-compare scripts — so you can gate every new quant before it enters your pipeline this weekend. Pin artifacts that pass by digest rather than a mutable tag, since tags can be re-uploaded under you, and re-quantize from the source checkpoint with a calibrated recipe if no known-good build exists.