Edge0 runs a 35B model in 3GB of RAM, streaming experts off SSD
A 35B-class model on a 24GB Mac at ~fp16 quality — Apache-2.0 and #1 on Hugging Face, though Apple-only and shaky at tools. Plus: Scry makes search queryable.
Copy markdown
Experts live on disk, not in RAM
Edge0-35B-A3B keeps all 256 experts on SSD in 4-bit and streams only the 4 a token routes to (~3B active params), so peak memory sits near 2.9 GiB. A trained prerouter prefetches the next experts to hide storage latency — up to 59% faster decode, landing 15-18 tokens/sec on a Mac mini M4 Pro. Translation: a 35B-class model on hardware that could never hold one.
Quality holds within ~4 points of fp16
Across five OpenCompass benchmarks the int4 build scores 79.2 versus 83.2 for the fp16 Qwen3.6 base it distills from — a 3.9-point gap, with AIME 2026 at 86.6, HumanEval 90.9 and MMLU-Pro 81.0. A Recover-LoRA pass claws back most of the quantization loss. That's good enough to be a real local daily driver, not a demo.
The catch: Apple Silicon only, weak at tools
Today it runs solely through MLX on Apple Silicon — GGUF/llama.cpp support is still an open feature request (issue #29061), so no Windows or CUDA path yet. The preview is also weak at agentic and tool-use work. Reach for it as a local chat and reasoning model now; it isn't ready to be your agent backend.
Elsewhere: Scry turns web search into a SQL query
Scry exposes a 500 TB web index in ClickHouse that you hit with read-only SQL and Datalog instead of a black-box ranker — a transparent alternative to Tavily and Exa for agent research. It's congestion-priced: free for non-commercial use whenever there's spare capacity, metered at peak. A weekend build if you want your agent's search to be inspectable.