Open-Jev lands within a point of closed Jev — MIT and self-hosted

The open typed-decision model hits 95.5% on JevBench — within a point of closed Jev — runs local with no tariff, and now reads images too.

Nowline SEP 22 12:00 PM banner

Top AI stories from the last hour

Top AI stories from the last hour

Copy markdown

  • Within a point of closed Jev, and open

    On JevBench v1, the openjev-sglang build (Qwen3.6-35B-A3B) scores 95.5%, trailing TypeSafe's closed Jev 1.13 at 96.3% and GPT-5.6 Luna at 97.1% — a gap that sits inside the confidence bands. The project's own Open-Jev-9B reports 97.54% test and 91.97% out-of-distribution accuracy on 26,452 held-out decision rows.

  • No per-decision tariff

    Closed Jev is the cheapest hosted option at $0.027 per 1,000 decisions; the open builds carry no tariff at all when you self-host, at roughly 0.65s median latency — the same tier as the fastest systems in the benchmark. If typed decisions sit anywhere in your eval, grading or moderation loop, that line item can drop to zero.

  • Typed decisions, three interfaces

    Open-Jev returns structured probabilities instead of text through Choice (odds per candidate), Noul (a 0-1 condition scale) and Score (rubric grades), trained on 408,884 typed rows across 25 task sources. It never generates tokens, so the answer is a number you can threshold and the output is free. Build this weekend: swap your LLM-as-judge grader or reranker for it.

  • It reads images, not just text

    The MIT-licensed openjev cross-encoder (Qwen3.5, in 0.8B / 4B / 35B-A3B sizes) scores image claims too, lifting multimodal-claim accuracy from 0.52 to 0.84 — enough for zero-shot visual moderation or reranking without wiring a vision LLM into the loop.

  • The catch: pin the exact repo

    'Open-Jev' is really several efforts under one name — zefan-cai's 2B and 9B (a 27B is still training) and AlexWortega's Qwen3.5 line — built on different bases, and JevBench v1 is a 242-decision pilot. Check the size, base model and license of the specific weights before you route production traffic through them.

  • Elsewhere: Dream-RSI replays instead of recomputing

    Google DeepMind's Dream-RSI lets a discovery agent improve by replaying past attempts against a learned world model instead of re-running expensive searches, cutting agent calls up to 162x with the weights frozen. The code is open. Build this weekend: bolt a replay buffer onto your own search agent.