An open-source project called Strata published a claim last week that would have read as a typo in 2024. Qwen 3.8 Flash Next, a model with 125 billion parameters, running on a single RTX 4090 at roughly a hundred tokens per second, which is about as fast as a competent person types when they already know what they want to say. One consumer graphics card, the kind that lives in a gaming PC, plus ordinary system memory. The architecture is what makes it possible: Flash Next is a mixture-of-experts model, so only a small share of those 125 billion parameters activates for any given word it produces, and the rest can sit in commodity RAM instead of the expensive fast memory soldered onto the card.
Read conventionally, that is a story about the frontier arriving at the desk, and the conventional read is correct as far as it goes. Builders on r/LocalLLaMA reported comparable numbers two days earlier on a 5090 that had been power limited. Then there is the other post from the same forum this week. A builder documented his path from one 3090 to twenty DGX Sparks, and the first hard limit he reported hitting was electrical. The breakers in his house gave out before VRAM, interconnect bandwidth, or model support did.
That second post is the more useful artifact of the two, and it names a frame most of the coverage is not applying. Local AI is a premises problem. Premises in the sense the utility industry uses the word: the building, the circuit, the meter, the lead time on a thing you cannot download. The compute frame trains you to read capability as a number that falls every quarter, because for fifteen years it has. Premises numbers are fixed instead by a service panel installed in 1974, by a landlord, by a commodity market, and by a county permitting office.
Three things become visible immediately through that frame. The first is the gap between nameplate draw and sustained draw. A benchmark measures a card at its peak for the length of a benchmark; a house measures everything plugged into it, continuously, and the whole value proposition of a local rig is that it runs the overnight batch, the background indexer, the work you do not want leaving the building. The second is amperage as a hard unit. Twenty amps on a residential circuit is twenty amps, and no firmware update has ever found another four. The third is memory supply. Micron CEO Sanjay Mehrotra said this week that memory will be considerably tighter in 2027 and 2028 than it is now, and the strategy local inference has just converged on is buying a great deal of ordinary RAM to hold experts that are not currently firing.
The frame is hard to see because the premises bill has been somebody else's for twenty years. What a hosted inference provider actually sells, underneath the per-million-token price, is a building: the substation, the chillers, the land, the memory bought on contract eighteen months ago at a price nobody has to disclose. The API price is a premises cost with the premises removed from view. That is the product. Local inference moves that same cost onto a wall you own.
This pattern has a long run of prior instances, and they resolve the same way. In the middle 1990s the gating factor on putting a server in your own office was the wiring closet, the cooling, and whoever had to sign for the circuit, rather than the server. By 1998 the colocation business had been reclassified in practice from real estate into power delivery, because power and cooling were what customers were buying when they rented a cage. I was reading that coverage as it ran, and the part that stuck was how fast the conversation moved off the machines. Within one cycle, nobody discussing Exodus was discussing racks.
The rhyme repeats at every scale. Processor clock speeds stopped climbing around 2005 because of watts, not because of lithography, and the industry rewrote itself around parallelism to get out of the way of a thermal limit. Google sited its original Oregon data center on the Columbia River for access to hydroelectric power, and Facebook picked North Carolina for the same underlying reason stated differently: cheap, reliable grid power. Geography became a function of the grid. Crypto miners went to dams and flare stacks for the same reason, and the leftovers of that migration turned up in this week's signal pool, where somebody implemented the Qwen3.5 architecture in FPGA fabric to serve quantized models on used mining boards. That hardware exists because a previous generation chased cheap electricity.
The frame also re-reads the consumer end of the week, which looks unrelated. Platformer's coverage of OpenAI's DevDay put Dots, the company's new always-on agents, in a line running back to Bill Gates predicting in 2023 that apps were finished and you would simply tell your device what you wanted. Set aside whether that is right. Notice the word always-on, borrowed from utility vocabulary to describe a software launch. An agent whose defining feature is that it never stops has a duty cycle near one. Every product direction the industry announced this autumn converts compute from a spiky load into a baseload, and baseload is the thing premises either have or do not.
The same constraint shows up at the hyperscale end wearing better clothes. The Rundown's read on Gemini 4 Argon was that the leaderboards look strong and you probably cannot use it yet, a capacity constraint stated in the language of a product announcement. The software side keeps handing out free improvements that make the same point from the other direction: a dual-Spark operator reported better than fifty percent more throughput this week purely from moving to GLM 5.3 Flash, with no new silicon involved. That cadence is what makes the premises frame easy to miss. Software improves weekly. A service panel improves when an electrician has an opening.
The number that would show this frame had landed is tokens per watt, and almost nobody posts it. Local benchmark threads report decode speed, context length, and quantization format, which are the measurements the compute frame asks for. The measurements the premises frame asks for are sustained watts under a realistic duty cycle, amps per box at the panel, and the price of a DDR5 kit on the day you need it rather than the day you planned for it. The thing that breaks first is whichever of those you did not price.
Which leaves the question the next two years answer. Every previous time the premises asserted itself, centralization won, because a building is the one cost that amortizes better at scale than at home: the closet became the colo, the colo became the cloud, and each step spread one building's electrical cost across more tenants. The current wave is running that tape in reverse, with models finally small enough to come home. Nobody yet knows whether a sparse 125-billion-parameter model on a gaming card is a strong enough reason to out-argue a service panel, and the twenty-Spark house has already filed the first notice.