Explorative Modeling: train on best-of-K, generate in one step
A UIUC-Harvard method turns exploration into a third scaling axis, matching diffusion quality with less inference compute. Code is out; plus Kimi K3 on 29GB.
Copy markdown
Push multimodality into training, not inference
Diffusion factors generation into hundreds of denoising steps; Explorative Modeling instead generates K candidates each training step and trains only on the best match to the target. Multimodality gets handled during training, so at run time the model emits an image, action, or token sequence in a single forward pass — 1 inference step where Diffusion Policy needs 100, at matching success rates.
The receipts: 1.43 FID, ~300x faster to converge
On ImageNet 256x256 it lands a near-SOTA 1.43 FID with no guidance and converges roughly 300x faster than the standard SiT recipe, at 6.2x better sample efficiency and 4.1x better FLOP efficiency. Robotics (RoboMimic), a Maze2D world model at 80x fewer steps, and masked-diffusion LMs all show the same one-/few-step parity.
Exploration as a third pretraining axis
The authors frame best-of-K exploration as a scaling knob alongside data and parameters: the compute-optimal amount of exploration keeps rising the longer you train, and its payoff grows with scale (13% to 23% with model size, 7% to 36% with data). Translation for builders: another dial to convert compute into quality.
It's a drop-in best-of-K loop, code's on GitHub
Adding exploration is a short best-of-K training loop with pseudocode in the paper, open-sourced at github.com/alexiglad/XM. If you train image, video, robotics, or diffusion-LM models, this weekend you could bolt it onto an existing recipe and test one-step sampling yourself.
Elsewhere: the 2.8T Kimi K3 now runs on 29GB of RAM
A community build gets Moonshot's Kimi K3 — the largest open-weight model yet at 2.8 trillion params — running on about 29GB of RAM at 0.50 tok/s via aggressive quantization. Painfully slow, but it means the biggest open model on Earth is technically self-hostable on a beefy workstation, not just a datacenter.