llama-modchip runs a 284B DeepSeek on 64GB of RAM, no GPU

An Apache-2.0 llama.cpp patch fits a near-frontier MoE on a desktop at ~10 tok/s — while Google doubles TPU inference density and Goldman pulls Claude Code.

Nowline NOV 10 6:00 AM banner

Top AI stories from the last hour

Top AI stories from the last hour

Copy markdown

  • A 284B model on a 64GB desktop

    llama-modchip quantizes a 284B DeepSeek-V4-Flash MoE to ~1.71 bits (56.6 GiB) and runs it entirely in CPU RAM — no GPU, no swap — at ~43 tok/s prefill and 9–15 tok/s decode on a single Ryzen 9. It's Apache-2.0, ships a patched llama.cpp server plus prequantized GGUFs on Hugging Face, and stock llama.cpp, Ollama and LM Studio reject the format, so you need its binary.

  • Google doubles TPU inference density

    Google launched TPU 8i Max — two 8i chips per module, doubling compute, SRAM and HBM bandwidth — aimed at the memory wall in long-context and multi-agent inference, plus an 8tx variant with hardware-level confidential inference for regulated teams. It also open-sourced TPU Sync, which tiers KV cache off-chip to cut time-to-first-token by 90%+, with vLLM and SGLang backends. No cloud pricing yet.

  • Goldman pulls Claude Code from its traders

    Goldman Sachs restricted Claude Code and GitHub Copilot to roughly 12,000 engineers, cutting off the traders and bankers who'd adopted them — a signal that big regulated shops are reining in who gets agentic coding tools, and how. Multiple outlets reported it; Goldman hasn't spelled out its reasoning.