Grammar files cut reasoning tokens in half — scaffold tricks keep outpacing model upgrades.
Top Signal
Structured CoT: Grammar Files Cut Reasoning Tokens by 50%+
workflow
r/LocalLLaMA
A developer published a technique using llama.cpp grammar files to constrain chain-of-thought output into a structured format — forcing the model to reason in compact key-value pairs instead of verbose prose. The result: equivalent answer quality with dramatically shorter reasoning traces, meaning faster inference and lower cost on local hardware. This matters because reasoning models (Qwen3.6, DeepSeek V4) burn most of their tokens in the thinking phase. By defining a GBNF grammar that enforces structured reasoning steps, you skip the filler without losing the logic. The technique works with any llama.cpp-compatible model today. If you're running local reasoning models, test a grammar-constrained CoT format against your current prompts — you may halve your token budget with no quality loss. This is another data point in the emerging pattern: scaffold design beats model size.
Read more →
Fast Signals
OpenAI Ships Dedicated PII Detection Model
new tool
HN Front Page, r/LocalLLaMA
OpenAI released a purpose-built privacy filter model for detecting and masking PII in text. Unlike regex-based approaches, it handles contextual PII (names in conversation, implicit identifiers). If you're building anything that processes user data through LLM pipelines, this slots in as a pre/post-processing guard — evaluate it against your current PII scrubbing before it becomes a compliance expectation.
Link →
DeepSeek V4 Architecture: 10x KV Cache Reduction at 1M Context
platform change
r/LocalLLaMA, HN Front Page, GitHub Trending
Community analysis of the V4 tech report reveals the architecture cuts KV cache from ~50GB to ~5GB at 1M context length. This isn't just a benchmark story — it's a concrete architectural advance in MLA (Multi-head Latent Attention) that makes million-token inference practical on fewer GPUs. Flash variant is already showing Haiku-replacement-tier tool calling in production.
Link →
Wuphf: Markdown+Git Wiki Layer Your Agents Maintain Themselves
new tool
HN Show
A Show HN project implementing the Karpathy-described pattern of agent-maintained wikis. Uses BM25 (bleve) + SQLite indexing over a local ~/.wuphf/wiki/ directory — no vector DB. Agents read and write markdown, git tracks history. If you're building multi-agent systems that need shared persistent memory, this is the simplest architecture worth testing.
Link →
Qwen3.6-27B Hits 80 tps on Single RTX 5090 with NVFP4+MTP
workflow
r/LocalLLaMA
A reproducible recipe using vLLM 0.19 with NVFP4 quantization and Multi-Token Prediction achieves 80 tokens/sec with 218k context on one consumer GPU. This is the kind of deployment config that makes local frontier-tier inference viable for production. The HuggingFace weights are already published.
Link →
ds2api: Drop-in Middleware Converts DeepSeek to OpenAI/Claude API Format
new tool
GitHub Trending
Trending on GitHub — a lightweight Go middleware that wraps DeepSeek's client protocol into OpenAI, Claude, and Google API-compatible endpoints. Supports multi-account rotation and deploys via Docker or Vercel Serverless. Useful if you're evaluating DeepSeek V4 but your stack assumes OpenAI-format APIs.
Link →
kreuzcrawl: Rust Crawling Engine with 11 Language Bindings
new tool
r/LocalLLaMA
Open-source structured data extraction engine designed for reliability across languages — Python, Node, Go, and 8 others. Built in Rust for performance. If you're building RAG pipelines or need to crawl at scale for training data, evaluate this against your current Playwright/Scrapy setup.
Link →
Radar
PaddleOCR-VL-1.5 Runs Book OCR via llama-server
A walkthrough of running PaddleOCR's vision-language model through llama-server for local book digitization. If you need offline OCR that understands layout and context, this is a zero-API-cost pipeline worth bookmarking.
Link →
GLM 5.1 Running Locally at 40 tps
Zhipu's GLM 5.1 is showing strong local inference numbers — 40 tokens/sec generation with 2000+ prompt processing. Another option in the increasingly competitive local model space, worth benchmarking against Qwen3.6 for your use case.
Link →
ik_llama.cpp Seeking Vulkan Contributors
The ik_llama.cpp fork (known for dramatically faster CPU and CUDA inference) is actively recruiting Vulkan backend contributors. If you run local inference on AMD GPUs, this project is worth watching — or contributing to.
Link →
Convergence Watch
qwen 3.6
TRENDING
14 mentions across r/LocalLLaMA, HN Front Page, GitHub Trending
Seven consecutive days of multi-source coverage. Today's signal is shifting from benchmarks to production deployment recipes — NVFP4 quantization, KV cache optimization, and head-to-head comparisons with DeepSeek V4 Flash. The model is crossing from 'impressive' to 'default local choice' territory.
deepseek v4
TRENDING
10 mentions across r/LocalLLaMA, HN Front Page, GitHub Trending
Third consecutive day across 3+ sources. Discussion maturing from launch hype to architecture analysis — the 10x KV cache reduction and Flash variant's tool-calling quality are the durable signals. DeepEP library trending on GitHub indicates infrastructure teams are already building on it.
claude code ecosystem
TRENDING
4 mentions across GitHub Trending, HN Show
Skills directories (mattpocock/skills, claude-code-templates, awesome-codex-skills) are trending on GitHub simultaneously. The pattern is clear: agent skill/prompt sharing is becoming its own ecosystem layer, decoupled from any single agent tool.