virtio-nvgpu: CUDA and NVENC in a KVM guest, near bare-metal
One card serves many KVM guests at 98-100% of bare metal by forwarding ioctls, not APIs. Plus an 8MB on-device tool-caller and a daily LLM value map.
Copy markdown
Near-native NVIDIA GPU inside a VM, no full passthrough
virtio-nvgpu forwards driver-level ioctls between guest and host instead of translating graphics APIs, so CUDA, NVENC encoding, and Vulkan/OpenGL all run inside headless KVM guests. On an RTX 3060, GPU-bound work hits 98-100% of bare metal, and four guests sharing one card sustain ~103 fps combined — about a single guest's throughput, split evenly. You can sandbox GPU agents or run multi-tenant inference on one card without VFIO locking the whole GPU to a single VM.
An 8MB tool-caller that runs on a Raspberry Pi
Cactus's Needle 3 — surging up Hugging Face's trending list this week — is a 121M-param model shipping as 8-29MB binaries that does function calling, structured extraction, and embeddings entirely on-device. It decodes at 400-4k tokens/s on a Raspberry Pi 5, beats models 10x its size on mobile tool calls, and after a short fine-tune its 4-layer version edges out DeepSeek V4 Flash on function-calling. Apache-2.0, and one weight file covers every depth from 2 to 20 layers — enough to drop cloud round-trips for on-device agents.
A daily map of which model is worth its price
bestmodelforyourbudget plots every model's Artificial Analysis intelligence score against its blended API cost and draws the "value frontier" — the models where nothing cheaper is also smarter. Filter by a minimum score to find the smartest model you can afford, with coding and math sub-scores, tokens/sec, and time-to-first-token alongside. Free, open source, and refreshed daily.