PrismML's Bonsai 2 crams Qwen3.8-27B into 5.9GB at 98% quality
Ternary weights fit a 262K-context, tool-calling model on a laptop — plus Microsoft's open GPU stack and agents that can now run your smart home.
Copy markdown
A 27B model at 98% of its quality, in 5.9 GB
PrismML's Bonsai 2 27B compresses Alibaba's Qwen3.8-27B to ternary (-1, 0, +1) weights — about 1.76 bits each — and holds 98.2% of the original's benchmark average (83.9 vs 85.4). It's Apache 2.0, so you can ship whatever you build on it.
Fast enough to be a daily driver, at 262K context
It hits up to 143 tok/s on an RTX 5090 and 44 tok/s on an M5 Max, carries a 262K-token window, and takes text-plus-image input with agentic tool calling. That's a real local backend for coding agents and computer-use flows, not a demo toy.
The catch: it needs PrismML's llama.cpp fork
The low-bit kernels aren't in stock llama.cpp — install PrismML's fork, or an Ollama / LM Studio / vLLM build that bundles them. Pull the GGUF (PTQ1_0 at 5.95 GB) for CUDA and CPU, or the MLX 2-bit build for Apple Silicon.
Microsoft open-sources TauGrid for your GPU cluster
TauGrid (MIT, v0.4.2) is a Kubernetes-native stack to run, schedule, and monitor training, fine-tuning, and inference on GPU clusters: tau CLI, Kueue fair-share queueing, KubeRay for distributed Ray, GPU health auto-drain, and checkpoint resume. One Helm install and you submit jobs as YAML instead of gluing five tools together.
Your AI agent can now run your smart home
Google opened Home MCP in early access: any MCP client — Claude, ChatGPT, Gemini — can monitor devices, read camera history, and control Nest and Matter gear, with guardrails that block sensitive actions. It needs a Google Home Premium Advanced ($20/mo) plan. A clean weekend build: an agent that actually runs your house.
Elsewhere: open models still own Hugging Face
The trending board stays stacked with runnable open releases — Qwen3.8-27B on vision-language, MiniMax-H3 for near-commercial image-to-video, and YuE2-3B for music with agentic editing — while GGUF quants keep out-downloading the base weights.