Xiaomi's DFlash hits 3,000 tok/s on a 1T MoE; EAGLE lands in llama.cpp — local AI's best day in weeks.
Top Signal
Xiaomi DFlash: 1,000–3,000 tok/s on 1T MoE model, open-source promised
platform change
r/LocalLLaMA
Xiaomi's Tilert infrastructure is now serving MiMo V2.5 at 1,000–3,000 tokens/second on a 1-trillion-parameter MoE using a custom DFlash persistent-kernel and fused sparse-routing execution. The blog post details both the architecture and measured throughput across hardware configs. This matters because it validates that MoE inference at truly interactive speeds is achievable at production scale — not just on toy benchmarks. The DFlash approach uses persistent thread blocks to eliminate kernel launch overhead across sparse expert routing, avoiding the cold-start penalty that typically tanks MoE throughput. Open-source release is explicitly promised 'coming soon.' Bookmark the Xiaomi MiMo blog now; when it drops, this becomes an immediate candidate for any vLLM/SGLang stack running sparse models. If your inference infra touches MoE architectures, this is the most important release to watch this week.
Read more →
Fast Signals
EAGLE speculative decoding confirmed merged into llama.cpp mainline
new tool
r/LocalLLaMA
After trending for two days, EAGLE speculative decoding is now in llama.cpp mainline. Add a compatible draft model to your config and pull latest — this is a free 2–4x throughput upgrade on consumer hardware with no quality regression. No experimental flags, no custom builds required.
Link →
Pyodide 314.0: any Python package can now publish a WASM wheel to PyPI
platform change
Simon Willison
Pyodide's latest release removes the custom build-pipeline requirement — standard PyPI publishing now works for WASM-compatible packages. Simon Willison immediately shipped luau-wasm 0.1a0 as a proof of concept. This collapses the gap between server-side Python AI code and browser-executable Python, making client-side inference tooling and no-install AI apps significantly easier to ship.
Link →
Why GPTQ 4-bit doesn't destroy perplexity — math derived from scratch
research to practice
r/LocalLLaMA
A practitioner worked through GPTQ's weight-compensation step from first principles: quantize one weight, then adjust all remaining weights via inverse Hessian to absorb the error — that's why 4-bit GPTQ loses almost nothing vs naive rounding. Directly actionable for format selection: if you're choosing between Q4_GPTQ and Q4_K_M for a production deployment, understanding this helps you reason about which workloads tolerate which tradeoffs.
Link →
Rio de Janeiro's 'homegrown' LLM exposed as a rebranded model merge
emerging signal
HN Front Page, r/LocalLLaMA
Two independent sources today — HN and r/LocalLLaMA — flagged that the city-backed 'Rio 3.5' model is a merge of Nex 2.5 PRO with no novel training; Nex-AGI filed the GitHub issue with receipts. Directly actionable: add base-model lineage verification to your model evaluation checklist before integrating any 'institutional' or 'government' LLM release. Similarity evals against known base models catch this.
Link →
Heretic Grimoire: distributed, takedown-resilient model weight preservation
emerging signal
r/LocalLLaMA
A new project targeting model availability loss — content-addressed, multi-node local-first storage for LLM weights designed to survive DMCA-style removals. Emerged directly from the Claude Fable export-control wave. Not a production tool yet, but the architecture pattern is worth understanding as model availability becomes less reliable for builders depending on specific capability tiers.
Link →
Dual YOLOv8n at 42 FPS on RK3588S NPU — no GPU, open code
new tool
HN Show
Working demo of two YOLOv8n models running in parallel at 42 FPS on the Khadas VIM4's RK3588S NPU using a multithreaded pipeline. Code is on GitHub. Direct reference implementation for builders targeting edge inference on ARM NPU silicon — increasingly the default chipset in AIoT and embedded devices.
Link →
Radar
Gemma 4's encoder-free arch may enable skip-STT speech-to-speech
Community exploration of using Gemma 4 12B's encoder-free architecture to bypass the ASR/STT bottleneck entirely for voice pipelines. No working implementation yet, but the hypothesis is technically plausible and could eliminate a full latency tier in voice AI stacks — worth watching for the first working demo.
Link →
aisuite gains OpenCoworker — desktop agent layer on multi-provider SDK
Andrew Ng's aisuite (unified interface across 20+ LLM providers) added OpenCoworker, a desktop AI agent built on top of it — currently trending on GitHub. Interesting pattern: multi-provider abstraction layers are growing agentic surfaces upward rather than staying as pure routing shims.
Link →
Convergence Watch
eagle3 speculative decoding
TRENDING
3 mentions across r/LocalLLaMA, GitHub Trending
Eagle3 has appeared 3 consecutive days (Jun 12–14) and the signal has fully resolved: the PR is merged into llama.cpp mainline. This is no longer experimental — it's a production upgrade available today. Any llama.cpp user should pull latest and add a compatible EAGLE draft model to their config for free 2–4x throughput.
claude fable 5
TRENDING
6 mentions across Simon Willison, HN Front Page, r/LocalLLaMA
Fable 5 has trended across 4 of the last 5 days. Today's signal is now downstream and ecosystem-level: the Heretic Grimoire preservation project and the 'built 80 mini-games before shutdown' HN post are community reactions, not new model news. The builder story has shifted from 'model access lost' to 'distributed preservation infrastructure emerging in response.'
rio de janeiro llm / model attribution fraud
2 mentions across HN Front Page, r/LocalLLaMA
Two independent sources today caught a government-backed model as a rebranded merge with no original training. As model releases proliferate, attribution fraud is emerging as a pattern builders need to defend against — base-model similarity checks should be standard practice before integrating externally-branded LLMs.
STALE: Latent Space newest item is >48h old