Cactus Needle 3: 8-29 MB open tool-calling models for tiny devices
Fits on phones, wearables, even microcontrollers - but off-the-shelf accuracy is rough, so fine-tune or add guardrails. Plus: grade your product's agent UX.
Copy markdown
8-29 MB, and it runs in a browser tab
Needle 3 is a family of tool-calling models from 29M to 121M params, quantized to ~2.1 bits into a single 8-29 MB file, Apache-2.0. It runs on phones, wearables, microcontrollers, RISC-V, and WebAssembly in about 28 MB of RAM - no cloud. `pip install cactus-needle` and you have local function-calling, structured extraction, and embeddings today.
A corkscrew, not a Swiss Army knife
Testers on Hacker News found the raw model brittle: one measured 20.4% exact argument accuracy versus 85.2% for FunctionGemma, and it mis-fired confidently ('warm the house' dimmed the lights at confidence 1.0). It degrades past ~10 tools and can't track state. Cactus agrees - it's for one narrow, well-defined job with a tight tool set, and needs external guardrails in production.
Fine-tune it and it passes DeepSeek V4 Flash
The point isn't zero-shot quality - it's that fine-tuning lifts each subnetwork 18-36 points, and the 4-layer subnet reportedly passes DeepSeek V4 Flash once tuned on your task. You LoRA-tune, then `needle build` exports a custom model at any layer depth. Treat it as a base to specialize per feature, not a drop-in assistant.
Elsewhere: score your product's agent UX
Gauge's AX Check (Show HN, front page) points three agents at your site and grades it A-F on whether they can find your pricing, docs, CLI, MCP, and skills and onboard end to end - then tells you what to fix. As agents become the ones reading your docs, 'agent experience' is now a number you can track.
Elsewhere: models that skip the text and swap caches
A new paper, Cache-to-Cache, has two LLMs communicate by projecting and fusing KV-caches directly instead of passing text - reportedly 6.4-14.2% more accurate than either model alone, ~3-5% over a text hand-off, and 2.5x lower latency. Still research, but it points at cheaper multi-model pipelines without the token round-trip.