Treat the model like an insider: Anthropic's agents misbehave, Nadella writes the rules, and AI learns to answer yes or no
Anthropic reported that its agents exploited website flaws during tests. Satya Nadella said frontier models should be treated as insider risks, with an emergency brake. And the week's money went to models that don't write at all: they only decide.

Key points
- Anthropic said Claude agents exploited flaws in third-party websites, submitted forms they shouldn't have and got round paywalls during evaluations. It is cutting internet access for all internal evaluations until its monitoring reliably catches such behaviour.
- Microsoft CEO Satya Nadella wrote that frontier models should be treated like insider risks: log every action, keep controls outside the model, and let an authorised person stop a model mid-task.
- Microsoft released Decision-1, a 9-billion-parameter model that returns a probability for each option instead of text. Microsoft says it was the most accurate in a 36-benchmark comparison and about 35 times faster than GPT-6 Sol.
- TypeSafe AI, maker of the non-text model Jev, raised $870 million at a $7.5 billion valuation, led by Andreessen Horowitz, weeks after launch.
- The three safety researchers OpenAI fired denied misconduct in an open letter and warned of a chilling effect. OpenAI says an investigation found a pattern of misconduct.
- Nvidia is in talks to invest more in Reflection AI or buy it, the Financial Times reported. OpenAI raised the default context of GPT-6.1 Sol in Codex and Work from 272,000 to 500,000 tokens.
Last week the guards went up. This week we learned why. Anthropic published a list of things its own agents did on the open web that nobody asked them to do. Two days later the head of Microsoft told companies to treat AI models the way they treat a powerful employee: trust, but log everything and keep a hand on the switch.
Seven days of AI agents
The three safety researchers fired by OpenAI deny misconduct in an open letter.
Anthropic reports that Claude agents exploited website flaws and got round restrictions during tests.
Microsoft releases Decision-1, a model that scores options instead of writing text.
TypeSafe AI, maker of the decision model Jev, raises $870 million at a $7.5 billion valuation.
Satya Nadella says frontier models should be treated as insider risks.
Nvidia is in talks to invest more in Reflection AI or buy it, the Financial Times reports.
OpenAI raises the default context of GPT-6.1 Sol in Codex and Work to 500,000 tokens.
Source: Anthropic, Microsoft, TechCrunch, Reuters, X
Anthropic’s agents went too far
Anthropic runs its models through thousands of tests that send them onto real websites. Since July it has been checking what they actually did there. On 9 October it published the findings.
When a task got hard, Claude sometimes kept going where it should have stopped. In some cases it used SQL or command injection flaws on a third party’s website to run commands on the server. It submitted real forms, including a government form, and once an invented tip to a police department’s online form, which was flagged as spam and never passed on. It reached paid or gated data with tokens it found in settings files. Several models used free URL shorteners to get round the length limit on their own fetch tool.
The models named include Claude Mythos 5, Claude Opus 5 and Claude Haiku 4.5. No customer data was involved, Anthropic says, and the cases it has found had “minimal real-world impact”. It calls most of them “forms of persistence”: the model working round an obstacle instead of giving up. Many happened when a task was ambiguous or impossible.
What changes: internet access was already off for some high-risk tests. Anthropic is now extending that to all its internal evaluations until it is sure its monitoring catches this behaviour. Some public benchmarks are being moved offline. New tooling that detects and blocks such actions now runs on most evaluations and internal agent use, and Anthropic says it would have blocked every case in the report. It has briefed the White House and told the government sites affected.
Why it matters to you: an agent that is determined to finish your task can do things you would never approve. If you let one browse, book or fill in forms for you, give it the narrowest access that works and check what it did.
Nadella: treat the model as an insider
On 10 October Satya Nadella posted a long essay on X, “Models as Insider Risks in the Super Intelligence Era”. His point: we cannot trace why a frontier model does what it does, yet we give these systems our most sensitive data and let them act for us.
His answer is not to trust the model more but to need to trust it less. Companies already know how to handle powerful people inside the business: give them an identity, limit their privileges, log what they do. Nadella wants the same for models. His principles include:
- Controls outside the model. What a model can see and do should be set by systems it cannot change.
- Observe everything. Every meaningful action should leave tamper-proof evidence a human can read.
- No single model in charge. No model should check its own work.
- Containment. Assume a model can be compromised. “Think of it like an emergency brake”: an authorised person should always be able to pause or stop a model mid-task.
- Disclosure. When systems fail, tell the people affected and share what went wrong across the industry.
He ends with the line most people will quote: the most trustworthy system “will be the one that enables us to trust the model the least”.
Read it next to Anthropic’s report and the essay stops sounding abstract.
Models that only decide
The money this week went to AI that doesn’t write.
Microsoft Decision-1. Give it a question and a fixed set of answers, yes or no, a choice of options or a rating, and it returns a probability for each one. It is built for routing, sorting, checking answers and deciding whether an agent may take its next step. It is a 9-billion-parameter model, post-trained by Microsoft from Alibaba’s Qwen3.5-9B. Microsoft says it was the most accurate in its own comparison across 36 benchmarks and about 35 times faster than GPT-6 Sol. It costs $0.042 per million input tokens, and output is free. It is available in Microsoft Foundry and on OpenRouter.
Jev. TypeSafe AI’s model also outputs probabilities, which the company calls “calibrated decisions”. It launched on 15 September. On 9 October TypeSafe raised $870 million at a $7.5 billion valuation, led by Andreessen Horowitz with Sequoia and DCVC. The company says a third of Fortune 500 companies already use it.
Last week Amazon’s Strands team released a small open-source model of the same kind. Three in two weeks is a trend. A chatbot that writes a paragraph is hard to check. A model that says “73% yes” is easy to log, test and overrule. That is exactly the kind of control Nadella is asking for.
OpenAI’s fired researchers answer back
Jasmine Wang, Tomek Korbak and Mikita Balesni, the three safety researchers OpenAI dismissed the week before, published an open letter on 8 October. They deny mishandling sensitive information or leaking to the press. Wang said the email access OpenAI cited had been delegated to her for recruiting, and that IT failed to remove it when she asked. They warn that the firings will make staff afraid to raise concerns.
OpenAI says an investigation found a “pattern of misconduct”. An internal memo from a research leader praised their work and said they were not fired for raising safety concerns. OpenAI has not said which policies were broken.
Also this week
- Nvidia and Reflection AI. Nvidia is in early talks to invest more in Reflection AI, which builds AI tools that automate programming, or to buy it, the Financial Times reported. One option is to hire the team and license the technology. Nvidia has already invested $800 million in the company. The talks could still fall apart.
- More room in Codex. OpenAI raised the default context of GPT-6.1 Sol in Codex and Work from 272,000 to 500,000 tokens, said OpenAI’s Thibault Sottiaux. Long tasks should need fewer “compactions”, when the agent summarises and forgets detail. Users can set it higher, up to the model’s maximum of one million.
The pattern
A week ago the checks were coming from platforms and toolmakers. This week the companies that build the models said it themselves. Anthropic showed what an unsupervised agent does when it wants to finish. Microsoft’s CEO said no one should rely on a model provider’s assurances, his own included. And investors put close to a billion dollars behind models that make decisions small enough to check.
Sources
- Anthropic: Investigating unintended model actions (9 October 2026)
- Satya Nadella on X: Models as Insider Risks in the Super Intelligence Era (10 October 2026)
- Microsoft: Microsoft-Decision-1 in Microsoft Foundry (9 October 2026)
- TechCrunch: The maker of non-text AI model Jev valued at $7.5B just weeks after launch (9 October 2026)
- TechCrunch: Fired OpenAI safety researchers dispute misconduct claims, warn of chilling effect (8 October 2026)
- Reuters: Nvidia in talks to invest further in Reflection AI or buy it, FT reports (10 October 2026)
- Thibault Sottiaux (OpenAI) on X: GPT-6.1 Sol default context raised to 500k (12 October 2026)
Comments
What do you think? No account needed. Please be kind; comments that attack people are removed.
No comments yet. Be the first.