Your Windows inference server loses two-thirds of its throughput the moment you click away.
Top Signal
Unfocused console windows silently cut Windows inference speed 2-3x
workflow
r/LocalLLaMA
A dev chasing inconsistent benchmarks found that a Windows LLM server attached to a console window throttles hard the instant that window loses focus: an RTX 5090 running a ~27B NVFP4 model via ninfer held 130-200 tok/s while the terminal was focused and collapsed to ~50 tok/s when it wasn't. The cause is Windows' foreground scheduling and DWM behavior applying to the console host, not the model or engine. The fix is one line of process management: run the server detached/headless rather than in an attached console. This matters beyond one rig. Anyone who has benchmarked a local stack on Windows by starting a server in a terminal and then switching to a browser to test has been measuring the throttled path, which quietly poisons engine-vs-engine comparisons. Action: if you self-host on Windows, relaunch your server headless as a service or with a detached process and re-run your numbers before drawing conclusions.
Read more →
Fast Signals
1-bit 27B runs in the browser at 25-30 tok/s on a 6GB laptop GPU
new tool
r/LocalLLaMA
A WebGPU build puts a 1-bit-quantized 27B model in a browser tab with no install, hitting 25-30 tok/s on an RTX 3060 Laptop with only 6GB VRAM. Treat this as a distribution channel, not a quality claim: recent quant sweeps show 1-bit degrades badly, so use it where zero-install demo access beats output fidelity.
Link →
285B DeepSeek vision MoE served on consumer 3090s at 60+ tok/s
workflow
r/LocalLLaMA
DeepSeek-V4-Flash-Vision-Exp (285B MoE, 157GB weights) runs on 10-12x RTX 3090 using an SM86-compatible vLLM build with FP4 experts plus FP8 attention, decoding 60+ tok/s with DSpark speculative decoding at k=3. The reusable part is the recipe: SM86 vLLM plus mixed FP4/FP8 keeps Ampere viable for frontier-size MoE.
Link →
Qwen3.8-Flash-Next lands on MLX-serve with 1M context
platform change
r/LocalLLaMA
The MLX-serve port ships with a 1M-token context window, making long-context work practical on Apple silicon without a CUDA box. If you have been sending million-token jobs to a hosted API purely for context length, this is worth re-benchmarking locally this week.
Link →
DeepSeek soft-retires V4 Pro without a formal deprecation notice
platform change
r/LocalLLaMA
V4 Pro appears to have been quietly pulled from active rotation rather than announced as deprecated. If you pin that model ID in production, verify it still resolves and pick a fallback now; soft retirements break callers with no changelog to point at.
Link →
GLM 5.3 Flash Q4 posts 60 tok/s decode, 550 tok/s prefill on M3 Ultra
emerging signal
r/LocalLLaMA
First solid Apple-silicon numbers for GLM 5.3 Flash at Q4, with prefill fast enough to make large-context agent loops tolerable on a Mac. Worth a head-to-head against Qwen3.8-Flash-Next if you are choosing a daily-driver local model for M-series hardware.
Link →
NVIDIA Cosmos3 64B gets INT4 quants for both CUDA and MLX
new tool
r/LocalLLaMA
INT4 builds bring a 64B image generation model into local reach on a single high-VRAM GPU or an M-series Mac. If your product pipeline still calls a hosted image API for internal or draft assets, this is the checkpoint to test bringing that in-house.
Link →
Qwen3-TTS finetune moves emotion control into inline transcript tags
research to practice
r/LocalLLaMA
Rather than passing a separate instruction parameter, this finetune accepts control tags inline in the transcript, so emotion can shift mid-utterance instead of per-request. The technique generalizes: if you are steering any TTS or audio model per-call, inline tagging gives finer control for the cost of one finetune.
Link →
Radar
Text prompts to playable synths, with the training run published
A solo dev trained an audio model that generates infinite one-shots and turns text prompts into playable synth patches, then released the weights, the inference pipeline, and a video walking through exactly how it was built. The end-to-end training walkthrough is the real artifact for anyone training a domain audio model.
Link →
Desert Ant Labs pitches on-device intelligence for every product
A new entrant aiming at embedded, on-device inference as a product primitive rather than a research demo. Unproven and light on public technical detail, but it lines up with a month of sub-1GB models running on phones and ancient hardware.
Link →
AMD Threadripper Halo Station targets the local-inference desk
AMD is now selling a workstation configured explicitly for large local models, which puts a vendor-supported option against the DIY multi-GPU rigs. Bookmark for pricing and bandwidth specs before your next hardware decision; the DIY numbers have been winning on GB-per-dollar.
Link →
escrcpy trends: a scriptable control surface for Android
A graphical scrcpy wrapper for displaying and controlling Android devices, trending on GitHub. Not an AI tool, but agent-native browser and desktop control harnesses have trended three days running, and this is the obvious substrate for the mobile equivalent.
Link →
Convergence Watch
ai research provenance
TRENDING
3 mentions across Simon Willison, r/LocalLLaMA
Second straight day across two sources. Terence Tao warns good open problems are being mined non-renewably; OpenAI's Navier-Stokes claim drew a r/LocalLLaMA charge of 'surveillance plagiarism' — that hosted providers train on users' research sessions and later present the results as unassisted. Unresolved, but the practical read for builders is concrete: if your prompts carry novel IP, check retention terms or keep that work on local weights.
qwen3.8
TRENDING
4 mentions across r/LocalLLaMA
Six of the last seven days, four mentions today: an MLX-serve 1M-context release, a Strix Halo quant-fitting writeup, an uncensored 27B GGUF finetune, and a 5 tok/s iGPU viability thread. Single-source, so discount the signal, but the shift is real — discussion has moved from 'is it good' to deployment logistics on specific hardware. It is the default local baseline now.
deepseek-v4
TRENDING
2 mentions across r/LocalLLaMA
Third day in a week. V4 Pro was quietly retired the same day someone published a working 285B V4-Flash-Vision serving recipe on consumer 3090s. Read together: DeepSeek is consolidating onto the Flash-Vision line, so build against that family rather than V4 Pro.
local hardware capex
TRENDING
5 mentions across r/LocalLLaMA
Threadripper Halo Station, Apple A20 Pro at ~115 GB/s, a 70GB VRAM Titan rig, Strix Halo quant fitting, and a top-voted 'do not let FOMO win' warning all landed together. The counter-post is the interesting part: the community is starting to push back on gear acquisition as capable models keep shrinking to fit hardware people already own.
SOURCE DOWN: HN Front Page returned 0 items
STALE: Latent Space newest item is >48h old