DeepMind's double-blind eval hides the test so a model can't peek
Neither side sees the other inside an encrypted enclave — a fix for the ~half of models that leak test data. Plus PIM inference gains and Hindi ASR.
Copy markdown
Neither side sees the other's cards
The pilot runs Gemini and the benchmark inside a Google Cloud Confidential Space enclave: the evaluator never touches the model's weights, and Google never sees the test prompts. On a single H100 it added under 5% overhead — cheap enough to run as a matter of course, not a stunt.
~Half of models show signs of test leakage
It targets contamination — a model that already saw the questions in training and then posts inflated scores. Research cited in the pilot flags leakage in roughly half of tested models, so the benchmark you pick a model on may be fiction. A cryptographic proof the provider never saw the test makes a score mean something again.
Outsiders hold the answer key, not Google
The whole point of the box is that Google can't tune to tests it can't see. MLCommons and the Singapore AI Safety Institute supply the private benchmarks, OpenMined's PySyft gates what code can run and blocks network calls, and only aggregate scores leave the enclave. First model graded: Gemini 2.5 Flash Lite, with GPU clusters next for larger models.
Elsewhere: Samsung puts a compute unit inside DRAM
At Hot Chips 2026, Samsung's LPDDR5X-PIM — logic baked into the memory die — ran AI inference 3.01x faster than plain LPDDR5X at up to 8x the effective bandwidth. It's drop-in mobile DRAM, so the payoff is faster on-device inference without bolting on a bigger, hotter accelerator.
Elsewhere: Hindi joins the Open ASR Leaderboard
Hugging Face added its first Indic language — Hindi, 500M+ speakers — with 4,888 speakers across 428 districts and 12 attributes logged per clip. If you ship speech, you can now see how Whisper Large v3 Turbo, Granite Speech, Voxtral Mini and VibeVoice hold up region by region instead of trusting one aggregate number.