Sakana's PC-ALM trains 1,000-layer nets without backpropagation
MIT-licensed JAX code is out: each layer learns from local error signals, landing within ~2 points of backprop at 1,000 layers — plus Moonshot's Kimi K2.8.
Copy markdown
Layer-local training, and the code is out
Sakana AI's PC-ALM (Augmented Lagrangian Predictive Coding) trains residual MLPs up to 1,000 layers to within ~2 points of backpropagation on MNIST — with no global backward pass. The JAX implementation ships MIT-licensed on GitHub, so you can run it today.
The trick: a PI controller per layer
Each layer carries its own Lagrange multiplier and alternates a local activation step with a dual step that accumulates prediction error — the authors call it 'a PI controller per layer.' That fixes plain predictive coding, which fades in deep, narrow nets: 77.75% vs 68.13% on Fashion-MNIST at depth 32.
Why builders care: no phase-locking
Because every layer updates from local signals, there's no network-wide forward-then-backward phase lock — pointing toward layer-parallel training and lower activation memory, the path to training deeper on the hardware you own. The catch: results stop at MNIST/CIFAR scale, and it's research code, not a drop-in trainer.
Elsewhere: Moonshot's Kimi K2.8 preview
Kimi K2.8 Preview surfaced on trackers, slotting between the K3 flagship and K2.7 Code with a 1M-token context and coding/agent quality reportedly near K3 but with leaner thinking. It's hosted-only this round — proprietary, no open weights.
Build this weekend: an inbox for background agents
AWS open-sourced Pizza Bot (Apache 2.0), a self-hosted inbox that runs agents asynchronously — start a task, schedule it, or fire it from a webhook, then triage results in Unread/Action queues. It plugs into Anthropic, OpenAI, Bedrock, Gemini, OpenRouter, or local Ollama.