Colibri runs 2.8T-param MoE models on consumer hardware, in pure C
The trick: weights stream off your SSD as they're needed, no GPU required. Plus a fully-local ElevenLabs, a token-cutting proxy, and agent web eyes.
Copy markdown
744B–2.8T params, on the machine you own
colibri (Apache-2.0, 28.5k stars) treats VRAM, RAM and NVMe as one tier and streams experts on demand, so Kimi K3 (2.8T) or GLM-5.3 (744B) run without full residency. Reality check: ~1.8 tok/s CPU-only on a 128GB desktop, 5.8–6.8 tok/s on 6× RTX 5090 — slow, but it runs where nothing else will.
It's a JIT, but for weights
Routing heat drives per-layer LRU caching, learned pinning and one-layer-ahead prefetch, with dual-SSD striping and a 57× smaller KV state — token-exact, no silent precision loss. Pure C, only Python for the launcher; CPU, CUDA, Metal and Vulkan backends. Build is `./setup.sh && make kimi_k3`.
A fully-local ElevenLabs, in 646 languages
VoiceStudio (AGPL-3.0, 28k stars) does zero-shot voice cloning from a 3–15s clip, video dubbing with speaker preservation, system-wide dictation and .m4b audiobooks — 16 TTS and 11 ASR engines, all on your box. Runs CPU-only from 8GB RAM; CUDA, Apple MPS/MLX and ROCm supported. Weekend build: a private dubbing pipeline with no per-character bill.
Cut coding-agent tokens 60–90% by compressing shell output
rtk (Rust, 80k stars) is a drop-in CLI proxy that filters and compresses command output before it reaches the model — one binary, 100+ commands, <10ms overhead. It trims output, not your whole bill, but for agents that live in the terminal that's the noisy part. Apache-2.0.
One CLI gives agents eyes on X, Reddit, YouTube, GitHub
Agent-Reach (Python, MIT, 75k stars) pulls web pages, YouTube, RSS, GitHub, X, Reddit and more through a single CLI with no per-platform API keys, and falls back to a backup backend when a site shifts its API. Handy for autonomous research and monitoring loops; scraped access stays brittle, so budget for upkeep.