A 78B Apache-2.0 model that wakes only 3.46B params, and a local-inference scene splintering into one-rig engines.
Top Signal
Kolibri-1: 78B Apache-2.0 MoE with 3.46B active params and 1M context
new tool
HN Front Page, r/LocalLLaMA
Aleph Alpha released Kolibri-1, a 78B-parameter mixture-of-experts model with only 3.46B active parameters per token, context up to 1M tokens, and an Apache 2.0 license. It landed simultaneously on HN's front page and r/LocalLLaMA, with a tech report and a third-party analysis. The active-parameter ratio is the story: a 4.4% activation rate puts 78B-class weights within reach of consumer hardware, where throughput tracks active params while VRAM/RAM tracks total. That is the same economics that made GLM-Flash-class models runnable on 128GB mini PCs. Combined with a permissive license and long context, it is a credible open substitute for mid-tier hosted APIs in RAG and document pipelines. What to do: pull the weights and benchmark against your current hosted mid-tier model on your own long-context eval set before quantized community builds arrive, and read the tech report for the routing details before assuming the 1M context holds up in practice.
Read more →
Fast Signals
Simon Willison: every usage-billed API needs a hard budget cap
workflow
Simon Willison
Willison argues that pay-by-usage services should ship default hard budget caps that return errors past $X/month rather than soft alerts. With agents issuing unattended API calls, the absence of a cap is now a product defect, not an edge case. Audit your own usage-billed endpoints for a kill-switch, and set provider-side caps on every key an agent can reach.
Link →
r/LocalLLaMA names the trend: inference engines overfit to one rig
emerging signal
r/LocalLLaMA
A widely-read post labels the proliferation of engines tuned to a single model-plus-hardware combination "overfit inference engines" — impressive headline numbers that collapse off the author's exact setup. Three separate posts the same day fit the pattern, plus a complaint thread about suspiciously uniform Strata praise. Treat any single-rig tok/s claim as unreproduced until you run it on your own hardware.
Link →
Ninfer 4080 squeezes 262 tok/s from a 16GB RTX 4080 at 100k context
new tool
r/LocalLLaMA
An individual dev released NInfer 4080, an engine targeting 16GB-class GPUs, reporting 2720 tok/s prefill and 262 tok/s generation on a 27B GSQ-quantized Qwen 3.8 at 100k context. That is a usable long-context coding model on a mainstream card rather than a workstation. Worth a bookmark if you have a 16GB card — and note it is exactly the single-rig tuning the overfit-engines post warns about.
Link →
Two 300B MoE models on one 128GB Strix Halo mini PC
workflow
r/LocalLLaMA
A detailed writeup runs GLM-5.3-Flash at ~580 tok/s prefill and MiMo-V2.6-Flash up to 44 tok/s decode on a single 128GB AMD Strix Halo box, using EXL3 weights and an open ROCm engine. It is the clearest non-CUDA recipe yet for serving 300B-class MoE from one machine. If you were sizing a multi-GPU build, price the mini PC path first.
Link →
MIT-licensed local call graph fixes grep-blind agentic refactors
new tool
r/LocalLLaMA
A dev shipped a fully local, MIT-licensed code knowledge graph for large Python codebases, built on the observation that grep finds mentions while refactors need actual callers — the gap that makes coding agents miss call sites. It runs with no cloud dependency. Useful as an MCP-style context source for agentic refactors where the agent currently guesses at blast radius.
Link →
A guide to RL across multiple agent harnesses, not just one
research to practice
r/LocalLLaMA
A long-form guide covers reinforcement learning against several agent harnesses rather than training to a single one, targeting the generalization failure where a model learns a harness's quirks instead of the task. This matters given earlier findings that identical local weights score very differently across Codex CLI, pi, and OpenCode. Read it before you fine-tune for agentic use.
Link →
Anthropic publishes an Opus 5.5 prompting guide for Claude Code
platform change
HN Front Page
An official guide to getting the most out of Opus 5.5 in Claude and Claude Code hit HN's front page with 141 points. Vendor docs rarely qualify as signal, but harness-specific guidance from the model's authors is the cheapest available tuning for anyone already shipping on Claude Code. Skim it against your current CLAUDE.md and system prompts.
Link →
Radar
FTL: a from-scratch OS aimed at cloud workloads
A new operating system built specifically for cloud environments, from the author of previous microkernel projects, with the repo already public. Worth watching as part of the broader push to replace general-purpose Linux assumptions in sandboxed and agent-executed workloads.
Link →
Sopro V2 Turbo: 120M-param voice cloning on CPU
An update delivering cleaner cloned voices at the same 120M model size and the same CPU-only speed. If you need TTS inside a product without a GPU budget or an external API, this is the class of model to evaluate.
Link →
Anyworld: local LLM as multiplayer RPG dungeon master
A self-hosted multiplayer text RPG where a local model runs the game. The interesting part for builders is not the game but the pattern: shared, persistent world state driven by one local model serving several concurrent users.
Link →
Convergence Watch
kolibri
2 mentions across HN Front Page, r/LocalLLaMA
The only genuinely cross-source item today: a release landing on HN and r/LocalLLaMA within hours, with a tech report and independent analysis already attached. Two sources is below the three-source bar, but on a four-source day it is the strongest independent confirmation available. Watch for quantized community builds as the follow-on signal.
local inference engines
TRENDING
4 mentions across r/LocalLLaMA
Four items today — Ninfer, Strata, the Strix Halo ROCm build, and SSD-streamed Qwen — plus a meta-post naming the pattern. Continues the local-hardware thread running since 2026-09-27 and the earlier llama.cpp fork-sprawl warning. Caveat: all of it is one source, so this is a subculture consolidating, not yet independent confirmation.
agent harness
TRENDING
2 mentions across r/LocalLLaMA, HN Front Page
Present on four of the last seven days across up to three sources, and today in both a multi-harness RL guide and Anthropic's Opus 5.5 guidance. The harness is settling in as a tunable layer of the stack in its own right — the same weights now produce materially different results depending on it.
qwen3.8
TRENDING
2 mentions across r/LocalLLaMA
Recurring since 2026-09-27 and again today as both the 176B Flash Next SSD-streaming run and Ninfer's 27B target. The 176B-on-16GB post closely repeats a 2026-09-27 item, so treat it as a continuing technique rather than news; the signal is that Qwen 3.8 has become the default substrate for local engine work.
SOURCE DOWN: HN Show returned 0 items