Edge0 runs a 35B model in under 3 GB of RAM on Apple Silicon
An Apache-2.0 MoE keeps its 4-bit weights on SSD and streams only the experts each token needs — 35B where a 7B fit. Plus Z.ai's cheap 1M-context API.
Copy markdown
35B intelligence, under 3 GB of RAM
Edge0-35B-A3B is a sparse MoE — 256 experts, 4 active per token — that keeps its 4-bit checkpoint on disk and pulls only the experts a token actually touches, so peak active memory is ~2.9 GiB on a Mac mini M4 Pro. That's a 35B-class model running where your old 7B did, decoding at 15–18 tok/s.
A prerouter hides the SSD latency
A trained prerouter guesses each token's experts one step ahead, overlapping storage reads with compute for up to +59% decode throughput. The catch: it's a preview, MLX/Apple-Silicon only for now, and 4-bit plus Recover-LoRA costs about 3.9 points versus the fp16 base (79.2 vs 83.2 average).
Z.ai's GLM-5.3 FlashX: 1M context at $0.37/$1.25
Zhipu's new FlashX variant (320B total, 18B active, hybrid sparse-linear attention) serves a 1M-token window at $0.37 in / $1.25 out per million tokens, clocking ~76 tok/s in practice against a 200 tok/s ceiling. It's API-only with no weights, but a cheap long-horizon pick for agent runs.