KoboldCpp 1.120 runs Qwen3.8-Flash-Next and Ling 3.0 locally
Open-weight MoE drops finally load in a mainstream local runner: 262K context from 64GB RAM, a DirectIO load mode, an Ollama MLX path — not llama.cpp.
Copy markdown
It finally loads in a real local runner
KoboldCpp 1.120 (Aug 29) merges full support for Qwen3.8-Flash-Next and Ling-3.0-Flash, with prebuilt binaries for NVIDIA, AMD ROCm, Vulkan and Apple Metal. The week's headline open MoEs now run in a mainstream GGUF runner instead of waiting on framework support.
64GB of RAM, not a datacenter
Qwen3.8-Flash-Next is 125B total but activates only 6B params per token, so unsloth's GGUFs run from 64GB of unified memory up — the ~85GB IQ4_XS quant fits by paging its n-gram table to SSD, and clocks ~36 tok/s on a 64GB M5 Max. Native context is 262K tokens.
DirectIO and combined mlock/mmap
A new --usedirectio model-load mode lands, and mlock and mmap can finally be combined — finer control over how weights sit in memory on tight-VRAM or slow-storage boxes. Small on paper, real when you're squeezing a 90GB file onto a laptop.
llama.cpp mainline still can't
Qwen3.8-Flash-Next support is still an open feature request (#27741) upstream in llama.cpp; the KoboldCpp fork shipped a working build first. On Apple Silicon, Ollama 0.33.1 added an MLX path for the same model on Aug 26.
Watch the quants
Maintainers warn that broken GGUF quants are circulating — pull the official Qwen or unsloth files, or you'll hit gibberish output or a failed load. The hybrid GDN-linear plus sparse-attention design is easy to quantize wrong.