Jeff has been training 0.8B decision models on consumer hardware at home and publishing the weights, which answer in roughly thirty milliseconds. The same day, a solo developer's ImaJev-4b, fifteen days of fine-tuning on text and photos, came in first of ninety-one entries on JevBench and scored above GPT-5.6 Luna on DecisionBench. Both results are self-reported against benchmarks the same community built, so the ranking carries a discount. What the ranking measures carries none.
The work these models do is classification: routing a request to one of four handlers, triaging inbound, approving or denying, pulling a field out of a document and judging whether it is valid. Every one of those has a bounded output space and a checkable answer. That is why a model small enough to train in a spare bedroom can do them, and why renting a frontier API for them has always been a slightly odd arrangement. A decision with four possible answers does not require a model that can also write a sonnet.
This has a shape I recognize. When AlexNet won ImageNet in 2012 by a margin that embarrassed every hand-engineered entry, most of the industry filed it as an academic result about labeling photographs. Neural networks had been written off for two decades, and what had changed was the compute rather than the theory, GPUs finally making a dismissed method trainable. The people who treated it as a leaderboard event underweighted it for years; the ones who read a general capability change started building. Benchmarks are where step changes tend to surface first, dressed as narrow technical wins.
The obstacle sits upstream of the model. In most deployed agent loops the decision steps do not exist as steps. One prompt goes to one large model, and that model retrieves, judges, and writes prose in a single pass, so the judgment leaves no separate trace. There is no log of what got routed where, no record of what got approved, no labels, and therefore no eval. An operator in that position cannot buy a cheaper decision model, because they cannot say what decisions their system currently makes.
Take a support inbox running one frontier call per message. The prose that goes back to the customer is generation. The choice of which queue the message lands in, whether it reads as a refund request, whether it needs a human: those are three decisions wearing one invoice. Nothing in the output tells the operator how the model reached them, or whether the third one has been wrong since June.
Simon Willison's write-up of a Muse agent transcript prices that gap in a form anyone can understand. The agent's auto-reply told a courier 'yep I'm here.' Nobody came down. He waited twenty-three minutes, left, and logged a negative rating. Read as a classification problem the failure is small and specific: no step in that loop distinguished an acknowledgment from a claim about the physical world. The distinction is binary, checkable, and well inside what a sub-1B model handles. That step did not exist in the loop, which is why nothing caught it.
Jensen Huang's NVIDIA comes at the same position from the other side with OpenShell, giving local and open agents process, filesystem, and network boundaries enforced by the runtime instead of by sentences in a system prompt that the model can reason its way around. More than a hundred firms joined the surrounding safety work; OpenAI did not. Both projects point at one move: the checkable parts come out of the prose and into code that fails loudly.
Dario Amodei's Anthropic shipped Sonnet 5.5 the same afternoon, priced identically to Sonnet 5 and more than thirty percent faster. Generation getting cheaper on a vendor's schedule is fine. Judgment moving on a vendor's schedule is a different exposure, and we argued here last Tuesday that an eighteen-megabyte trained cache can change what a model refuses while the version string never moves. A small classifier on hardware in the building changes when its owner changes it, and the diff is readable by the person who owns the consequences.
Pulling the decisions out produces something nobody in this week's substrate is selling: a labeled record of the judgments a system makes on its owner's behalf. A thousand logged routing calls with outcomes attached is a private eval set, generated by ordinary operation, and it beats any published leaderboard because it is measured on the only distribution that matters. It is also the only honest check on a self-reported benchmark, Jeff's included, since the useful question is how those weights behave on one operator's traffic rather than on ninety other people's.
The work scales down further than it looks. A solo operator's decision steps are usually two, what gets an auto-reply and what gets escalated to a human, and an afternoon of logging covers both. A five-person team's decision steps are already scattered across three prompts owned by three people, so the first job is an inventory: which model is being asked for a judgment, on whose behalf, and who would notice if the answer drifted. That inventory is a morning's work, and most teams running agents in production do not have it written down anywhere.
Jeff's weights are eight hundred million parameters and answer in thirty milliseconds, on hardware most teams already own. The one that hands them last month's routing calls has a number by Thursday. The one waiting for a paper keeps paying per token for judgments it never wrote down.