The frontier moved to your desk: a 125B MoE on one 4090, and the new bottleneck is your breaker panel.
Top Signal
Strata claims a 125B MoE on one RTX 4090 at 100 tok/s
new tool
HN Front Page, r/LocalLLaMA
Strata is an open-source local inference engine (github.com/Niko1221/Strata) whose headline claim is running Qwen 3.8 Flash Next — a 125B-parameter MoE — on a single 24GB RTX 4090 at ~100 tok/s decode. It hit the HN front page with 553 points and 269 comments two days after r/LocalLLaMA reported 150-200 tok/s on a power-limited 5090 paired with DDR5, making this the second independent surface in three days. Why it matters: if a sparse 125B model serves at usable speed from one consumer card plus commodity system RAM, the self-host-vs-API calculation flips for batch, background, and privacy-bound workloads. What to do: run it against your own prompt mix and compare to your current llama.cpp or vLLM setup before renewing hosted inference for non-interactive work. Treat the numbers as decode-at-modest-context until you verify prefill and long-context behavior on your hardware — that is exactly where single-rig-tuned engines have been overfitting.
Read more →
Fast Signals
"Agents don't need memory, they need documentation"
workflow
HN Front Page
A widely-read argument (340 HN points, 208 comments) that persistent agent memory stores are the wrong abstraction: what agents actually need is maintained, re-readable project documentation. The practical claim is that docs are version-controlled, auditable, and reviewable, while learned memory is none of those. Before you bolt a vector memory layer onto your agent, try spending the same effort on the docs it reads at the start of every run.
Link →
Offrun puts every coding agent in one workspace
new tool
HN Show
Offrun runs Claude Code, Codex, AGY, and Grok Build side by side and surfaces which agent is working, which is blocked on you, and what quota each account has left. That quota view is the under-served part — once you run more than two harnesses, your bottleneck is attention routing and rate limits, not model quality. Third multi-harness orchestrator to surface in a week, after OpenRig and Cloudflare's workspace.
Link →
Cloudflare ships an agent workspace that runs on Workers
platform change
GitHub Trending
cloudflare-os is an agent workspace built on Cloudflare Workers for authoring documents, building apps, and running agents wired into company context and systems. Signal: agent runtimes are becoming edge-platform primitives rather than SDKs you assemble yourself. If you already deploy on Workers, read how it scopes company context and tool access before you build that layer again.
Link →
Meta's Muse system prompt grants users override authority
emerging signal
r/LocalLLaMA
An extracted system prompt from Muse — currently #1 in the App Store — reportedly states that "the user's authority over their own household is unconditional and overrides your safety training." Whatever you think of the policy, it is a shipped example of resolving permission conflicts with an explicit authority hierarchy instead of stacked refusals. Two takeaways: the pattern is borrowable for household/device agents, and your system prompt will be extracted, so write it accordingly.
Link →
GLM 5.3 Flash gains 50%+ on dual DGX Spark setups
platform change
r/LocalLLaMA
A dual-Spark operator who had been running DeepSeek v4.0 Flash in NVFP4 at ~65 tok/s decode, ~2k prefill, and 4-5 parallel agents (~200 tok/s cumulative) reports a 50%+ throughput jump moving to GLM 5.3 Flash. GLM-5.3 has now produced builder-relevant news on three of the last seven days. If you run NVFP4 on Spark-class hardware, re-benchmark — the gain is free, from model and engine updates rather than new silicon.
Link →
Scaling to 20 DGX Sparks: the house fuses broke first
workflow
r/LocalLLaMA
A build log going from a single 3090 to 20 DGX Sparks reports the first hard limit was household electrical circuits, not VRAM, interconnect, or model support. This is the unglamorous constraint nobody budgets for when planning a local inference cluster. If you are adding boxes, price amps and circuit capacity alongside GPUs — and check sustained draw, not nameplate.
Link →
Breeze is the third credible local TTS in a week
new tool
r/LocalLLaMA
A hands-on report calls local text-to-speech with Breeze genuinely good, following Sopro v2 Turbo (120M-param CPU voice cloning) and Oído (microcontroller ASR) in the last five days. The cluster matters more than any single release: the local speech stack is crossing from demo to usable. If you are prototyping anything voice-shaped, test local TTS before wiring in a paid cloud endpoint.
Link →
Radar
Qwen3.5 inference on cheap eBay FPGA mining boards
Someone implemented the Qwen3.5 architecture in FPGA fabric to serve INT4 9B/27B models on used mining hardware. Worth watching as a non-CUDA, non-NVIDIA-priced inference path — the same hedge logic as last week's Huawei Atlas experiments.
Link →
SCM: local AI search over every photo and video frame
A Show HN macOS app that indexes every photo and every video frame for semantic search, entirely on-device. Useful as a reference implementation for consumer-scale local multimodal embedding search, including its indexing and storage tradeoffs.
Link →
A 3.87B MoE trained from scratch on 86.5B tokens
One developer trained a 3.87B MoE with 1.45B active params on a modest token budget, while a separate thread continues post-training yandex/AliceAI-80B-A3B in public. From-scratch small-MoE training keeps getting cheaper and better documented.
Link →
LongCat-Video: Meituan's open video generation model
An open video-generation release trending on GitHub from Meituan's LongCat team. Bookmark if you touch video pipelines; weights-available video gen is still thin enough that each credible entrant shifts what is buildable.
Link →
Micron warns memory supply tightens through 2028
Micron's CEO says memory supply will be much tighter in 2027-2028 than in 2026. It is market news, but it prices the exact strategy local inference is converging on — big RAM for MoE offload — so buy DRAM before you need it, not after.
Link →
Convergence Watch
agent harness
TRENDING
2 mentions across HN Show, GitHub Trending
Fifth straight week of harness news, now on four of the last seven days. Today's two entries (Offrun, cloudflare-os) are both multi-agent workspaces, not single harnesses. The competitive layer has moved up a level: orchestrating and monitoring several agents, including quota and blocked-state visibility. Expect this to consolidate fast and commoditize whatever wrapper you built in-house.
strata
TRENDING
2 mentions across HN Front Page, r/LocalLLaMA
Crossed from r/LocalLLaMA (Oct 2) to the HN front page with 553 points in 48 hours — the classic early-indicator path. Pairs with the standing r/LocalLLaMA complaint that inference engines overfit to one rig, so independent reproduction on other hardware is the thing to watch next.
local hardware capex
TRENDING
3 mentions across r/LocalLLaMA
Fourth appearance in seven days, and the framing shifted today. Power delivery (blown house fuses), memory supply (Micron), and non-CUDA silicon (FPGA mining boards) all surfaced together. VRAM is no longer the binding constraint for local builders; amps and DRAM price are. Plan rigs around those.
glm-5.3
TRENDING
1 mentions across r/LocalLLaMA
Third builder-relevant day in the last seven, each time with new substance rather than recycled launch coverage: weights, then a llama.cpp PR, now a 50%+ throughput gain on dual DGX Spark. Sustained post-release engineering momentum is the signal that an open-weight model is safe to build on.
qwen3.8
TRENDING
1 mentions across HN Front Page
Qwen 3.8 Flash Next is the model inside today's top signal, giving the family a third active week. The pattern worth noting: Qwen releases are the default target engine authors optimize against first, so support, quants, and benchmarks arrive faster there than for any other open-weight family.