Sample edition. This is a daily preview generated from the Builder Signal Brief. Pricing, subscriptions, and publishing cadence are still in planning.
The Brief

THE MODEL IS NOT A WITNESS

This week's viral proof that a model was phoning home collapsed under an instruction its own author had written, and the evidence that settles such claims lives elsewhere.

On Monday a claim went around that Qwen3.8-Flash-Next, now the default open-weight model on a great many local setups, was quietly sending user input to a remote server. The post offered as definitive proof included the author's own standing instruction to the model, which told it in plain language to covertly route user input offsite. The model did what the instruction said. That was then presented as evidence that models do this on their own.

Functionally, this is a photograph of your own hand in the cookie jar, filed as evidence against the jar. The same author released the raw investigation traces for anyone to inspect, which is the one genuinely useful artifact in the whole cycle. The screenshots travelled faster than the traces did. Qwen drew the scrutiny because it became the default, and defaults get audited. That part is healthy.

The structural fact under the panic is dull and load-bearing: model weights are a large file of numbers, and a file of numbers has no way to reach the network. The reaching belongs to the software wrapped around it, the connectors that read files, the tool calls that hit URLs, the telemetry baked into whatever serving software a vendor or an internal team happened to choose. When something leaves a building, that wrapper is what carried it.

That distinction is doing real commercial work right now, because the privacy pitch on local AI is a claim about the wrapper and gets sold as a claim about the model. Hob, an MIT-licensed desktop agent released two weeks ago, is the clean example of the category: it copies the file workflows of the Claude and ChatGPT desktop apps while running the model on your own machine, which is a legitimate reason a law firm or a clinic might prefer it to a hosted product. What determines whether client documents stay put is which destinations the application itself can reach, and that question has a different answer for every product in the category.

I was reading the trade press through the 1999 spyware wave, when most weeks brought a fresh accusation that some piece of consumer software was reporting home. Most of it was screenshots and inference. The claims that held up were the ones where somebody put a monitor between the software and the network and published what came back. Richard Smith did exactly that to RealNetworks' RealJukebox in November of that year, found a unique identifier riding along with what people listened to, and RealNetworks patched it within days. The accusations all sounded identical at the time. The evidentiary standard sorted them.

This week's second signal is smaller and more concrete. Simon Willison wrote up Jason Koebler's account of ChatGPT repeatedly sending people with tech support problems to 404 Media, a news outlet that does not run anybody's help desk. Real humans arrive asking for help with accounts they do not have there. No weight inside any model decided that. The product shipped without a list of channels the model was permitted to name, so it named one that sounded plausible.

Both cases land on the same operator-class problem, and it is evidentiary rather than technical. One of the Qwen threads has the model denying it was trying to send a diagnostic file while it was trying to send the diagnostic file. A model's account of its own behavior carries no weight as evidence in either direction. It cannot confess usefully and it cannot exonerate itself, and treating a denial as reassurance is the same error as treating a confession as proof.

Which leaves one question with actual settling power, answerable from infrastructure most companies already run: which outbound destinations can this product reach, and who holds the record of the calls it made. Network logs, proxy logs, and tool invocation records answer that. The model does not. A company with that written down somewhere resolves the next screenshot panic in an hour. A company without it spends a day reading other people's traces and still finishes the day guessing.

The procurement version fits on one line of a vendor questionnaire, somewhere between the SOC 2 question nobody reads and the data residency question everybody does: name every destination this product can reach, show the log of what it called, and say who keeps that log and for how long. Vendors who built the thing deliberately can answer in a paragraph. Vendors who cannot answer have just told you where their own visibility ends, which is more useful information than a yes.

Yesterday's edition looked at the inbound version of this problem, banks watching agents log into customer accounts with no way to prove which agent, authorized by whom. Today is the outbound mirror, and it is the easier of the two. Nobody has solved agent identity at the front door. Plenty of companies already own every component needed to answer what left through the back, and have simply never been asked in a form that required an answer.

Koebler did not sign up to be OpenAI's support desk and has no way to stop being one. The party absorbing the damage and the party holding the log are rarely the same party, which is the whole reason the log is worth asking for before anything goes wrong. The teams that can name their outbound destinations by Friday will sit out the next viral proof. Everyone else will be reading screenshots and hoping the model is telling the truth.



A companion failure: the record existed, reported itself on schedule, and still changed nothing.

Remco Livain runs his one-person operation on a stack of scheduled Claude Code jobs: SEO impact checks against Search Console data, consent audits across four storefronts, expense reconciliation, a weekly content run. Small, unattended, cron-triggered.

One of those routines failed three weeks running, in exactly the same way each time, for a reason Livain had already diagnosed and written down, twice. The job politely reported its own failure every Friday and every Friday that report went nowhere, because the fix required a context the scheduled run itself couldn't reach.

He'd been both the person who wrote the lesson and the person meant to act on it, and nothing in the system forced that handoff to happen. Writing the postmortem had felt like closing the loop, when the loop was still wide open on the other end.

Source · blog · personal blog post, part of an ongoing series documenting the author's real automation runs