Ollaya: run Jev-style decision models locally, free and open

An open runtime puts decision models on your own GPU at ~8ms and $0.00003 a call. Plus Docker's cloud agent sandboxes and Anthropic starts billing refusals.

Nowline SEP 26 7:00 PM banner

Top AI stories from the last hour

Top AI stories from the last hour

Copy markdown

  • Ollaya: decision models on your own machine

    Ollaya is "Ollama for decision models" — pull and serve Laya, Decider, NLI and GLiClass locally behind an API that mirrors TypeSafe's Jev, so the official Python SDK 0.7.1 runs unchanged against localhost. On an RTX 4090, Laya answers a five-question request in 8–10ms versus 236–276ms hosted, at roughly $0.00003 a call; it's Apache-2.0 and covers 100+ languages. The trade-off: Laya is weaker than hosted Jev on hard queries, so reach for it on speed and cost, not raw capability.

  • Docker ships Cloud Sandboxes and OCI "Kits"

    Docker moved its microVM agent isolation off your laptop: Cloud Sandboxes run the same policies locally or in the cloud, boot in low-hundreds of milliseconds, scale 1→16 vCPUs and run unattended for hours — a direct answer to Vercel Sandboxes. New "Kits" package an agent, its tools and its guardrails as one OCI image on Docker Hub and GitHub, and Docker is submitting the spec to the CNCF so it stays an open standard, not a proprietary format.

  • Anthropic starts billing for pre-output refusals

    Since Sept 24, the Claude API charges for refusals that fire before any output in three categories — bio, frontier_llm and reasoning_extraction — at normal request rates; refusals in other categories stay free. If your app sends borderline prompts in those areas, they'll now land on the bill. Separately, cache diagnostics left beta: the diagnostics field ships in every Messages response now, no special header required.

  • Decision models get their gold-rush weekend

    The Jev-style "decision model" — a classifier that returns typed, calibrated answers instead of prose — is suddenly everywhere: a one-function LLM wrapper, an openjev clone on Hugging Face, and "Jev Plays Pokémon Red," an agent that made ~8,000 game decisions for $1.21. If you've been firing a full LLM at yes/no or pick-one questions, this is the weekend to swap in a cheap, fast decision model instead.

  • Elsewhere: OpenAI DevDay is this Sunday

    OpenAI DevDay lands Sept 29 in San Francisco — historically the launch window for new APIs and models. If your stack rides on OpenAI, it's worth clearing an hour for the keynote.