Speculative decoding methods stack — 4.68x on real coding prompts — while llama.cpp's forks outrun upstream.
Top Signal
Stacking an n-gram drafter on DFlash 2 hits 4.68x on real coding prompts
research to practice
r/LocalLLaMA
A LocalLLaMA user spent three days benchmarking the DFlash 2 PR build in llama.cpp against every other speculative decoding method on Qwen 3.8 27B, using 100 real coding prompts rather than synthetic ones. DFlash 2 alone delivers 2.26x. The finding that matters: adding a single n-gram drafter on top pushes it to 4.68x, and up to 8x on repetitive edit-heavy cases — the two methods compose rather than compete, which is not what most people assume. This is the third independent DFlash 2 result this week and the first with a full method-by-method comparison. What to do: if you self-host a coding model, build the DFlash 2 PR branch and test the n-gram stack against your own prompt distribution before assuming the numbers transfer — gains scale with how repetitive your output is. On hosted APIs, ignore this until you price out self-hosting.
Read more →
Fast Signals
MCP publishes a roadmap — read it before you build glue code
platform change
HN Front Page
The Model Context Protocol team laid out where the spec is heading, drawing 170 points and a heavy comment thread on HN. Read it before committing to custom tool-calling plumbing or workarounds the protocol is about to absorb natively.
Link →
One RTX 5090 runs Qwen3.8-27B at a real 262K context, 64.7 tok/s at 128K
workflow
r/LocalLLaMA
A reproducible vLLM recipe with the exact memory knobs written down: NVFP4 quant, the full 262,144-token window (not a truncated approximation), 77 tok/s short-context and 64.7 tok/s at 128K. If long-context agents are your bottleneck, this is a copy-paste starting point rather than another leaderboard number.
Link →
Your local model isn't dumb — your harness defaults are
research to practice
HN Front Page, r/LocalLLaMA
A Level1Techs writeup on how sampling and serving defaults, not weights, account for most of the quality gap people blame on local models. It landed the same day r/LocalLLaMA tore into Artificial Analysis's composite "Intelligence" score as meaningless. Audit your own config before switching models.
Link →
Apache Maka records every tool call and permission decision
new tool
GitHub Trending
A local-first agent workspace now incubating at Apache, storing model messages, tool calls, tool results, permission decisions and termination events as an append-only log. That audit-and-replay primitive is missing from most agent frameworks. Bookmark it if your agent runs need to be reviewable after the fact.
Link →
Ornith 1.5's broken MTP head gets fixed: +3% TPS, -33% wall clock
research to practice
r/LocalLLaMA
This closes the loop on midweek reports that Ornith 1.5 35B-A3B was slow with multi-token prediction. A user patched the head; tokens per second barely moved but total wall clock dropped a third. Another reminder that tok/s is the wrong metric — measure time to a finished answer.
Link →
The agent skill that matters is verification, not review
workflow
Simon Willison
Willison argues the core competence with coding agents is confidently instructing them and confidently verifying the result — sometimes line-by-line, often through other means. The same day, Torvalds credited an AI with the grunt work of a kernel debug session from hell while noting it flatly contradicted itself at points. Two very different builders converging: delegate the grind, own the verification.
Link →
OpenAI's Python SDK 3.x moves to httpx2 — audit your lockfile
platform change
Simon Willison
llm 0.33 upgraded to OpenAI Python 3.x and swapped httpx for httpx2, following an emergency 0.32.1 patch after the SDK's httpx drop broke fresh installs. If anything in your stack depends transitively on the OpenAI SDK's HTTP client, check your pins now instead of during an incident.
Link →
Radar
llama.cpp fork tuned for AMD GFX906 (Mi50, Radeon VII)
A user co-developed a HIP-optimized fork with GLM targeting the GCN-era AMD cards everyone else abandoned. Used Mi50s are cheap; if the performance holds, the cost-per-GB-of-VRAM math for a home rig changes materially.
Link →
Ninfer fork ported to CMP170HX doubles Qwen3.6-35B
Someone ported the Ninfer 3090 runtime to Nvidia's deliberately crippled CMP mining card and got roughly 2x llama.cpp throughput. Dead mining silicon is quietly becoming viable inference hardware — second such report this week.
Link →
OzBrain: a shared knowledge layer for agents and teams
A Show HN betting that agent-first chat replaces dashboards, so knowledge has to live outside any single tool. It's the sixth shared-agent-memory project in a week — the category is forming much faster than a winner is.
Link →
Terminal wattage monitoring for local inference rigs
A small tool for watching GPU power draw alongside your tokens in the terminal. Trivial on its face, but power-limiting is now a standard local-tuning knob and almost nobody instruments it.
Link →
OBLITERATUS trends: jailbreak corpora as a test suite
A widely-shared collection of model-bypass prompts is climbing GitHub Trending. The defensive read: treat it as an adversarial test set against your own guardrails, paired with Tencent's AI-Infra-Guard scanner from yesterday.
Link →
Convergence Watch
qwen3.8
TRENDING
3 mentions across r/LocalLLaMA, HN Front Page
Seventh consecutive day. The conversation has moved from 'is it good' to serving-config optimization — speculative decoding, NVFP4 quant, context-window recipes. That shift is the mark of a model settling in as default local infrastructure rather than a novelty worth benchmarking.
dflash2
TRENDING
2 mentions across r/LocalLLaMA
Third result in five days, each from a different user on independent hardware. Still confined to one source, so treat the multipliers as reproducible-within-one-community rather than validated. The moment to watch is llama.cpp merging the PR — that's when it becomes the default path.
llama.cpp
TRENDING
4 mentions across r/LocalLLaMA, HN Front Page
Four separate threads today: the DFlash 2 PR benchmark, a GFX906 fork, a CMP170HX port beating it, and a config critique. The interesting hardware work is happening in forks now — upstream has become the baseline everyone measures against rather than the thing they run.
agent memory
TRENDING
2 mentions across HN Show, GitHub Trending
OzBrain and Apache Maka today, following ai-memory, openviking, wildstatic and substrate over the past week. Six projects circling one problem: agents need durable, portable state that outlives a session and crosses tools. No standard is emerging yet — assume you'll rip out whatever you pick.