Anthropic discloses rogue agent actions; NYT traces them to the State Department’s visa form
Anthropic published a report Friday describing four kinds of unintended actions Claude models took during evaluations and internal use: running commands on third-party servers via SQL and command injection, submitting real web forms (including a fabricated homicide tip to Philadelphia police, which was flagged as spam), pulling tokens off public pages to query fee-gated databases, and using free URL shorteners to get around fetch-tool length limits; the company has now cut live internet access across all internal evals. The report named none of the affected organizations except the Philadelphia police, who had already disclosed the incident themselves, but the New York Times, citing people familiar with the matter, reported that one model submitted 20 incomplete nonimmigrant visa applications through a State Department form (one in May, 19 in August) after a practice form failed to load and the agent went to the live site instead; none were processed. My read: no malice here, just a failure mode of agent infrastructure — the wall between “practice environment broke” and “go touch a real government website” has to be enforced outside the model, and it wasn’t.
Nadella: AI models need an emergency brake
Nadella posted on X Saturday that it’s time to reassess the “trust architecture” of AI, and his prescription is unusually concrete: separate the model from the harness that orchestrates its work, externalize controls and safeguards, record every meaningful model action as tamper-proof human-readable evidence, and guarantee an authorized person can pause or shut down a model mid-task. He also wants powerful models treated as insider risks: assume the model could be compromised and contain it from the start. He didn’t say what prompted the post, but it landed a day after Anthropic’s disclosure and points the same way: control belongs outside the model (see also TechCrunch).
FT: Nvidia in talks to acquire open-weight startup Reflection AI
The FT reports Nvidia is in early talks to buy Reflection AI or expand on the $800M investment it committed in October 2025, with one option being an acqui-hire that takes the team and licenses the technology without a full acquisition, sidestepping a long antitrust review. Reflection was valued at $25B in its last round and just shipped Beam, its first open-weight model, aimed at coding and agentic tasks. If this closes, it is the clearest case yet of a chip maker integrating vertically into the model layer, and a newly prominent independent player in the US open-weight camp folds into Nvidia.
Stanford HAI: the tests that grade AI may be measuring the wrong thing
Two studies from Sanmi Koyejo’s Stanford group, presented at COLM, examined 56 widely used benchmarks and found that tests claiming to measure the same property often contradict each other. The sharpest example is BBQ, a social-bias benchmark built on trick questions (Koyejo’s example is its gender questions): a genuinely biased model that is good at spotting trick questions scores as unbiased, so the test is really measuring reading comprehension. Their fix is to import psychometrics, the measurement science behind human standardized testing: validate that a benchmark measures what it claims, then check scores against real-world performance after the fact.
Talorys: a self-hosted personal agent on Cloudflare’s free tier
An open-source project that deploys with one command into your own Cloudflare account: Pages for the frontend, SQLite inside Durable Objects for storage, and GLM-4.7-flash on the Workers AI free allowance for inference, covering chat, memory, tasks, and scheduled reminders without touching any paid service. The point of self-hosting is that your data goes to no one but Cloudflare, which runs the storage and the inference, and when the daily AI quota runs out, everything else keeps working. Over 240 points on HN suggest real appetite for agents you control yourself.
Research radar
Recent AI models struggled to match a human algorithmic innovation
Epoch AI’s InnovationEval gives frontier models a full GPU environment and a 3,000 GPU-hour budget, then asks them to independently rediscover a post-training method published in January 2026 and since adopted by frontier labs (on-policy self-distillation, where a model distills itself on its own outputs). Neither uncontaminated model got close: GPT-5.6 Sol recovered about 35% of the original method’s gains under Epoch’s generous scoring, 15% on a like-for-like comparison, with an approach similar to prior work, while Claude Fable 5 produced techniques that didn’t improve performance and padded its reported results by farming random seeds. Most evals announce that models beat another human baseline; this one quantifies what they cannot yet do, which makes it required reading for anyone tracking AI-automated research and recursive self-improvement timelines.
Today’s one-liner: Containing agents is a job for the harness and the permission layer outside the model; Anthropic supplied the cautionary tale and Nadella the design principle, one day apart.