transformers 5.22 adds FP8 LoRA: fine-tune big MoEs on one GPU
v5.22.0 cuts MoE adapter-training memory ~10x, adds Mistral Large 4 with per-request reasoning effort, and relocates an expert-parallel import that breaks code.
Copy markdown
FP8 LoRA, ~10x less backward memory
transformers 5.22.0 lets adapter training (LoRA via PEFT) run directly on FP8-quantized models — MoE backward peak memory at 8k tokens falls from ~28.9 GiB to ~2.8 GiB. That's the gap between renting a multi-GPU box and fine-tuning a large MoE on a single card.
Update: Mistral Large 4 lands in transformers
The release adds native Mistral Large 4 support (1T params, ~52B active, 1M-token context) with reasoning effort set per request — low/medium/high, default high; recommended sampling temp 1.0, top_p 0.95. You can load and serve it from stock transformers now, no custom code.
Before you upgrade: moved import, vLLM fix
Expert-parallel logic moved into its own file, so imports from the old path break — update them before you bump the version. The companion 5.21.1 patch fixes a rope_scaling setter bug that had broken remote-code loading in vLLM.