Mini-AGI: a byte-level model that keeps learning on one 8GB GPU
It trains from scratch on a laptop GPU and reads text in a nonstop batch-1 stream, holding 99.8% of what it learns — no fine-tune cycles, no forgetting.

Copy markdown
It reads, and it doesn't forget
Mini-AGI trains a language model from scratch on a single 8GB GPU, feeding it one continuous batch-size-1 stream of text and updating on every character it reads. The author reports it keeps 99.84% of its prior progress when it switches to a new subject — the catastrophic-forgetting problem that normally forces you to freeze a model or retrain from a checkpoint.
468M params, only 101M on the card
It's a byte-level mixture-of-experts — 256 raw byte values, a 4,096-character context — that grows and prunes its own experts as it learns. It currently sits at ~468M parameters but keeps only ~101M resident on the GPU, paging the rest to disk at ~2.4 GFLOPs per byte. That's how a ~470M-param model trains on an RTX 3070 laptop instead of a rented cluster.
The catch: no weights to download yet
This is one developer's research repo, not a packaged model. The code is public but the network is still on its first pass over the corpus, so there's no checkpoint to grab and every figure here is self-reported by the author. Treat it as an architecture to clone and train yourself, not a model to drop into production this week.
Build this weekend: a model that learns your stack
The unlock is a small model that absorbs your own data as a live stream — notes, a codebase, chat logs — and keeps improving without a fine-tune run or a RAG index in front of it. Clone the repo, point it at a folder, and let it read overnight on a gaming GPU you already own.