Tencent's 770B model just shrank 7.5x with ~98% of its performance intact — frontier weights now fit in a workstation.
Top Signal
Hy4-Preview compresses 1.5TB to ~200GB GGUF at ~98% performance
platform change
HN Front Page, r/LocalLLaMA
Tencent open-sourced Hy4-Preview (770B total / 49B active MoE), and today the number that matters landed: the 1.5TB checkpoint compresses to roughly 200GB in GGUF while retaining ~98% of measured performance. That moves a frontier-class open model from datacenter-only to a single high-RAM workstation — 256GB of system RAM plus a mid-range GPU for attention offload puts it in reach without an H100 cluster. The compression ratio is the story, not the model: sparse MoE checkpoints are mostly cold experts, and aggressive quantization plus expert offload is holding up at a scale where people expected a quality cliff. What to do: if you rent frontier inference for privacy-sensitive workloads, price a 256–512GB RAM box against your monthly API spend. Verify the quant yourself — this week's audit found 64 of 443 GGUF files mislabeled — and benchmark on your own task before committing hardware.
Read more →
Fast Signals
Qwen 3.8 27B hits 50 tok/s with 100k context on a 16GB GPU
workflow
r/LocalLLaMA
A full config walkthrough gets Qwen 3.8 27B fully resident in 16GB of VRAM at 100k context and 50 tok/s, using beellama.cpp — a llama.cpp fork most builders haven't touched. The recipe combines quantized KV cache with the fork's memory layout changes. Copy the config if you are sizing a single-GPU inference box; 100k context on consumer hardware changes what you can put in a prompt.
Link →
Curated llama.cpp PR list tracks CPU/RAM/disk hybrid inference
workflow
r/LocalLLaMA
Someone assembled the open llama.cpp PRs specifically touching CPU-only, system-RAM, disk-backed, and hybrid inference paths. This is where the large-MoE-on-cheap-hardware work is actually happening, months before it lands in a release. Bookmark it if you are running models bigger than your VRAM — several of these PRs are mergeable today against a local build.
Link →
Exo labs claims 4.8TB/s aggregate bandwidth clustering M5 Ultra Macs
emerging signal
r/LocalLLaMA
Exo reports 4.8TB/s of aggregate memory bandwidth by clustering M5 Ultra Mac Studios — roughly 4x a single unit's 1.2TB/s. Aggregate bandwidth is not the same as usable bandwidth for a single forward pass, so treat the headline skeptically until someone posts tok/s on a real model. Still worth watching if Apple silicon clusters are your alternative to GPU procurement at current prices.
Link →
Terminal Bench 4.0 puts GLM-5.3 within error bars of Fable 5
research to practice
r/LocalLLaMA
The new Terminal Bench release matters more than the leaderboard: it is one of the few evals that measures agentic shell work rather than single-turn answers. GLM-5.3, open-weight, lands statistically level with Fable 5 on it. If you are routing agent tasks by model tier, re-run your own harness against GLM-5.3 before you renew a frontier contract.
Link →
vphone-cli boots a virtual iPhone on Apple's Virtualization.framework
new tool
HN Front Page
A CLI that spins up an iOS guest using Apple's own virtualization stack — no jailbreak, no physical device farm. The obvious builder application is disposable, scriptable iOS environments for agent-driven UI testing and mobile automation. Worth an afternoon if you have ever hand-tapped through a mobile regression suite.
Link →
StemDeck runs open-source audio stem separation fully local
new tool
HN Front Page
Free, open-source, on-device separation of a mixed track into vocals, drums, bass, and instrumentals. For anyone shipping audio features, this removes a recurring per-minute API dependency and the associated upload of customer audio to a third party. Drop-in candidate wherever you are currently paying for hosted stem splitting.
Link →
Radar
Speculative decoding is visible to the naked eye at 2-3 t/s
A user running an MTP-equipped DeepSeek distill slowly enough to watch reports output surging exactly on predictable phrases like 'United States of America.' It is a free intuition for where draft-model acceptance actually pays off — and a reminder that your speculative gains are workload-shaped, not a flat multiplier.
Link →
DeepSeek flash v4 hits 67-84 t/s on two GX10s
Real throughput numbers for a paired small-form-factor accelerator setup, not vendor marketing. Useful reference point if you are evaluating GX10-class boxes against renting GPU hours.
Link →
Samsung details processing-in-memory silicon at Hot Chips
Compute moved into the DRAM itself, aimed squarely at the bandwidth wall that dominates inference economics. Nothing to act on today, but if PIM ships it changes the cost curve for memory-bound serving more than another GPU generation would.
Link →
typephp compiles PHP to native binaries
Swoole's AOT compiler ships PHP as a standalone binary, trending quietly on GitHub. Relevant mostly as a deployment-shape signal: single-binary distribution keeps spreading into runtimes that never had it.
Link →
Convergence Watch
hy4-preview
TRENDING
2 mentions across HN Front Page, r/LocalLLaMA
Second consecutive day across two independent sources, and the framing shifted from 'weights released' to 'weights are runnable.' When a release survives past the announcement cycle into practical quantization work, it is being adopted rather than admired. Expect llama.cpp support and community quants within days.
memory bandwidth
TRENDING
4 mentions across HN Front Page, r/LocalLLaMA
Samsung's PIM talk, Exo's Mac cluster claim, GX10 throughput posts, and the llama.cpp CPU/RAM PR list all attack the same bottleneck in one day — following HBM wafer-area and RAM price items earlier this week. The scarce resource in local inference is bytes moved, not FLOPs, and tooling is reorganizing around that.
qwen3.8
TRENDING
2 mentions across r/LocalLLaMA
Seventh straight day of mentions, but the character changed: today's posts are deployment recipes and a repair-shop anecdote, not release commentary. That transition from evaluation to routine use is the signal — treat Qwen 3.8 27B as a default local baseline rather than a candidate.
STALE: Latent Space newest item is >48h old