Ternary Bonsai 2 shrinks Qwen3.8-27B to 5.9 GB, keeps 98%

A ternary-quantized 27B that runs on a 16 GB laptop or in a browser tab, Apache-2.0 with vision. Plus Kimi K3 hits Bedrock and Copilot retires models.

Nowline SEP 20 12:00 PM banner

Top AI stories from the last hour

Top AI stories from the last hour

Copy markdown

  • 27B intelligence, 5.9 GB on disk

    PrismML's Ternary Bonsai 2 packs Qwen3.8-27B into ternary weights — about 1.72 bits each — for a 5.93 GB file that holds onto 98.2% of FP16 quality across 20 benchmarks. Coding even ticks up (89.42 vs 89.07). Apache-2.0, 262K context, with an optional 0.63 GB vision tower.

  • It runs where your old 7B did

    Decode speeds hit 46.8 tok/s on an M5 Max, 96.7 on a 4090, and 142.5 on a 5090, and it fits a 16 GB laptop or a 24 GB GPU. There's even a WebGPU build that runs the whole model in a browser tab.

  • The catch: bring PrismML's fork

    The GGUFs won't load in stock llama.cpp — you need PrismML's fork, and Mac users get an MLX 2-bit pack instead. Don't want to run it locally? It's already routable on OpenRouter.

  • What you can build this weekend

    A ~6 GB Apache-2.0 model with vision and tool-calling makes offline document-QA, on-device screenshot agents, and a fully private coding helper realistic on a laptop — no API bill, no rate limits, nothing leaving your machine.

  • Elsewhere: Bedrock and a Copilot cull

    Moonshot's Kimi K3 is reportedly live on Amazon Bedrock, giving AWS-locked teams a frontier open-weight option inside their VPC. Separately, GitHub's changelog flags several Copilot models for retirement in mid-October — check your pinned choices and migrate now.