GPT-6 Astra edits a small Qwen's weights to 6x its math, no training
An MIT researcher used a frontier model to patch 0.016% of an 0.8B model's weights in 50 forward passes, no gradients — and it only works on a smaller target.
Copy markdown
Qwen's math wasn't missing — its decoder was hiding it
On Qwen3.5-0.8B, Astra lifted MATH-500 from 7.9% to 48.1% and GSM8K from 13.8% to 67.1% — roughly 6x — with no fine-tuning at all. The tell: answers blowing past the 32,768-token limit fell from 93% to 33%, so the model was looping itself to death, not failing to reason.
131,072 weights, 50 forward passes, zero gradients
Astra traced the failure to one looping direction in MLP layers 15 and 19 and subtracted it from 64 output columns each — 0.016% of the model — using at most 50 forward passes and no training. If this holds up, you could repair a small local model's quirks with no GPU run, no dataset, and no gradient.
It removes a block — it can't add a skill
Point the same method at Gemma 3 270M and nothing moves (6.8%→7.0% on MATH-500), because that model rarely loops; its errors were ordinary, not one switchable component. The author's line: editing weights “could remove an obstacle that hid an ability, but it could not add an ability the model lacked.” No independent reproduction yet — full logs and code are on GitHub.
Weight-space joins prompts and code as a surface you can optimize
DSPy's Omar Khattab read the result as models getting “good at natively reasoning about and writing out neural network weights directly,” not just tuning prompts and harnesses. The near-term shape: a big model profiles the small model you actually deploy and patches it in place.