Cactus Needle 3 fits an on-device tool-calling agent in one file
The Apache-2.0 successor to Needle 2 does tool calls, extraction and embeddings offline — on a phone, a Pi, or in the browser.
Copy markdown
One file, no cloud, no key
Needle 3 is a single 9-35 MB weights file (121M params, 2.125-bit quant) that runs fully on-device — phones, watches, robots, air-gapped boxes, even the browser via WASM. On a Raspberry Pi 5 it decodes 400-4,000 tokens/s. Nothing leaves the device.
It punches 10x above its size
Cactus says Needle 3 beats models 10x larger on mobile tool calls and matches 2-3x bigger models on extraction; after LoRA fine-tuning, a 4-layer subnetwork reportedly passes DeepSeek V4 Flash. The same weights ship every depth from 2 to 20 layers, so you pick your own speed-versus-quality tradeoff.
The offline half of an agent stack
It handles exactly the local plumbing agents need — function/tool calling, structured extraction (messy text into typed fields), and text embeddings for on-device search and routing — with no network hop. A real unlock for smart-home, wearable, and robotics apps that can't assume a connection.
Confidence scores gate risky calls
New in v3: every tool call ships a 0-1 confidence score, so you auto-run above ~0.7, ask the user to confirm between 0.1 and 0.7, and refuse below. You tune autonomy to risk — a light switch versus a bank transfer — and measure it on frozen tests instead of guessing.
Build this weekend
`pip install cactus-needle`, Apache 2.0, one weights file plus sub-1 MB per-platform engines and a CLI runner (`./needle --model needle3.cact --tools tools.json`). Enough to wire a fully-offline voice-or-text agent into a phone or smart-home app tonight.