A 4B model, RL-trained, beats Postgres's own query planner
An open experiment: a distilled Qwen 4B with a 42MB LoRA cut join-query latency 44.7% on JOB — best-of-3, zero regressions. The code and method are public.
Copy markdown
The result: 1.81x on the Join Order Benchmark
Across 113 join-heavy JOB queries, picking the fastest of three model-suggested plans hit a 1.81x geometric-mean speedup — a 44.7% latency cut — with 68 wins and zero regressions against Postgres defaults. For repeated analytical queries, a tiny model is finding plans the built-in planner leaves on the table.
It's a distilled 4B with a 42MB adapter
The base is a distilled Qwen 3.8 4B; the only trained part is a LoRA of 21.2M parameters (42.5MB on disk). It emits planner hints rather than rewriting your SQL, so it sits beside Postgres instead of replacing it — small enough to run right next to the database.
How it was trained: SFT, then agentic RL
420 trajectories distilled from GPT-6 Astra bootstrapped the model via SFT, then agentic RL with a custom "anchored" GRPO variant ran 1,200 optimizer updates against real query timings. Hardware was modest: SFT started on a single RTX 3090, RL on a 2xH100 node at ~45 minutes per epoch.
The honest caveat: best-of-3 means three executions
The headline 1.81x comes from running three candidate plans and keeping the fastest. A single shot lands 1.41x geomean (1.29x total, 2 regressions) — still a win, but the big number assumes you can afford to try a few plans and measure. Best fit: hot, repeated queries where that cost amortizes.
Build this weekend: the harness is open
The full research harness — SFT, RL, and eval code plus JOB/IMDb/CEB task inventories — is on GitHub (polyphilz/qorl). Point it at your own workload, swap the base model, or adapt the hint format to another engine. A concrete template for doing RL on your own systems, not just chatbots.