On Friday, a 35-billion-parameter model started running agentic coding tasks at 79 tokens per second on an RTX 5070 Ti. That is a consumer graphics card you can buy at Best Buy for around $750. The model is Qwen 3.6-35B-A3B, released by Alibaba's Qwen team, and it uses a mixture-of-experts architecture that activates only 3 billion parameters at inference time while drawing on 35 billion parameters' worth of learned knowledge. Independent benchmarks from multiple users on r/LocalLLaMA show it outperforming Google's Gemma 4 26B on coding tasks. Several users report it is the first local model they have used as a genuine replacement for Claude or GPT API calls in daily coding workflows.
The interesting part is not the benchmark numbers. It is the economic structure underneath them. For the past three years, the AI industry has operated on an implicit assumption: serious inference requires serious infrastructure. You train on your own clusters or rent from hyperscalers, and you serve through API endpoints priced per token. The entire business model of the inference layer, from OpenAI to Anthropic to Google, rests on the premise that the gap between what you can run locally and what you can access through an API is wide enough to justify per-call pricing. Qwen 3.6 is the first model to make that gap feel narrow for a real, high-value use case.
Coding is not a toy benchmark. It is the single most commercially valuable application of large language models right now. When developers report replacing API calls with a local model for their daily coding agent workflow, they are not making a philosophical statement about open source. They are making a procurement decision. The math is straightforward: a one-time $750 GPU purchase versus ongoing API costs that, for heavy agent usage, can run $200 to $500 per month. The breakeven period is measured in weeks, not years.
This is not the first time a technology shift has played out through this exact economic logic. In the early 2010s, AWS made it possible for startups to access enterprise-grade infrastructure without buying servers. The structural advantage was clear: convert capital expenditure to operating expenditure, scale on demand, pay only for what you use. That advantage held for a decade. But around 2020, a countermovement began. Companies like Basecamp publicly documented how repatriating workloads from the cloud to owned hardware cut their infrastructure costs by millions. The pendulum swung because the gap between cloud and local capability had narrowed enough that the convenience premium no longer justified the cost for certain workloads.
Local LLM inference is entering its repatriation moment, but compressed into months instead of years. The mixture-of-experts architecture is the key enabler. Traditional dense models force you to run every parameter on every token. MoE models route each token through a small subset of expert networks, delivering the quality of a large model at the compute cost of a small one. Qwen 3.6 is not an incremental improvement on this idea. It is the first MoE model where the ratio, 35B total to 3B active, is aggressive enough to fit comfortably on hardware that millions of developers already own.
The ecosystem is responding in real time. Cloudflare open-sourced a tool called Unweight this week that achieves 15 to 22 percent lossless compression on LLM weights. Unlike quantization, which trades quality for size, Unweight produces bit-identical output in a smaller package. Applied to a model like Llama 3.1 8B on an H100, it saves roughly 3GB of VRAM. Applied to consumer hardware running Qwen 3.6, it means even tighter models on even cheaper cards. A community researcher separately published a technique using the Wasserstein metric to detect and correct tensor drift in quantized models, yielding measurably better quality at low quantization levels. These are not theoretical papers. They are tools you can download and apply today.
The configuration details matter and reveal how early this shift still is. Users report that a specific flag in llama.cpp, called n-cpu-moe, is critical for achieving full performance with Qwen 3.6. Without it, speeds drop significantly. The model handles 64K to 128K context windows on consumer hardware, but only with the right software stack. This is reminiscent of early cloud computing, where getting good performance required knowing which instance types to pick, which regions had capacity, and which networking configurations avoided bottlenecks. The knowledge barrier is real but temporary. It gets documented, automated, and eventually invisible.
What makes this moment structurally significant is the convergence of three independent trends. First, MoE architectures have matured to the point where the active-to-total parameter ratio enables genuine quality at consumer-hardware scale. Second, the tooling layer around local inference, from llama.cpp optimizations to lossless compression to better quantization metrics, is improving on a weekly cadence. Third, the use case driving adoption is coding agents, which happen to be the highest-value, highest-frequency application in the current AI landscape. When the most valuable use case becomes viable on local hardware, the incentive structure for the entire inference economy shifts.
None of this means API providers are in immediate trouble. Cloud inference still wins on convenience, on access to the largest frontier models, and on workloads that spike unpredictably. But the comfortable assumption that local inference is a hobbyist concern just broke. The next six months will likely see MoE architectures from other labs, Meta and Mistral chief among them, push the active-parameter ratio even further. The question worth watching is not whether local models can match API quality. For coding, that question was answered this week. The question is how fast the tooling and configuration complexity collapses to the point where choosing local is as easy as signing up for an API key.