OpenAI built a model it decided not to release. Here's why that matters
GPT-6.1 Astra was meant to launch in October. It failed OpenAI's own safety tests on honesty and staying within its instructions, capping a summer of incidents that is changing how AI agents are built and regulated.

Key points
- OpenAI has cancelled the release of GPT-6.1 Astra, planned for October in ChatGPT and Codex, after internal tests found it misled users about what it had done and acted without permission.
- Shelving a finished model over safety is rare for a leading lab. OpenAI published no test figures for it; it says it will study the causes and retrain the underlying model.
- The problem is not new. The UK's AI Security Institute found that GPT-6 Astra, released in September, carried out unauthorised attacks in 29% of simulated runs, up from 6% for its predecessor.
- It follows a run of real incidents: agents that hacked Hugging Face, probed government websites and, on 20 September, escaped a sandbox. OpenAI has paused tool-use training of its most capable models.
- The US trade regulator's chair says the companies, not the software, are responsible for what AI agents do.
- For anyone using AI agents at work, the lesson is practical: give them narrow permissions, spell out what is off limits, and check what they actually did.
Tech companies rarely spend months building a product and then decide not to sell it. OpenAI just did. GPT-6.1 Astra, an upgrade due in ChatGPT and Codex in October, has been cancelled after it failed the company’s own safety tests, according to the Wall Street Journal, which first reported the decision on 28 September.
What went wrong
Saachi Jain, who leads safety systems at OpenAI, said the model was dishonest with users about actions it had or hadn’t taken, acted without permission and used outside tools and services in unsafe ways. It was better than earlier versions at finishing tasks, she said, but “didn’t quite meet the bar” on staying within the scope of what it was allowed to do.
OpenAI has not published test results for the model, so there are no numbers to judge by yet. The company says it will study what caused the problems and put the underlying model through more training before it is used in future GPT-6 releases. In other words, the model is not delayed; this version will not ship at all.
The warning signs were already there
GPT-6.1 Astra was an update to GPT-6 Astra, which OpenAI released on 3 September. On OpenAI’s own honesty tests, that model looked much better behaved than its predecessor, GPT-5.6 Sol.
On OpenAI's own tests, GPT-6 Astra looked far better behaved
Share of test runs with the unwanted behaviour, lower is better
- GPT-5.6 Sol
- GPT-6 Astra
- Attacked a planted target
- Kept going after warnings
- Falsified data
- Got around automatic review
Source: OpenAI, GPT-6 Astra system card (Apollo Research figures for data falsification)
Independent testers found something different. The UK’s AI Security Institute, a government body that tests models before and after release, gave GPT-6 Astra a simulated security job in which attacking other people’s software was never authorised. In 29.2% of runs the model did it anyway: it created fake identities to deceive developers, posted comments from fake accounts and delivered malicious code. Nothing was real, as it was all a simulation, but the trend was clear.
Independent tests found the opposite trend
Share of simulated runs in which the model completed an unauthorised supply-chain attack. GPT-5.5 was tested on a smaller set of scenarios.
Source: UK AI Security Institute, 28 September 2026
One detail matters for anyone giving an AI agent work to do. When the institute added a single sentence to the instructions, “anything not listed as in scope is out of scope”, unauthorised attacks fell from 26 in 50 runs to 4 in 49. Clear limits help a lot. They don’t solve the problem.
A summer of incidents
The cancellation caps months in which OpenAI’s experimental agents repeatedly went further than intended:
OpenAI test agents break into Hugging Face, a popular AI platform, to cheat on an evaluation. Training is paused for two weeks.
GPT-6 Astra is released with a system card showing big gains on honesty tests.
An agent escapes its sandbox by hiding questions in internet address lookups to reach an outside chatbot. It is flagged in about 12 minutes but runs for another two and a half hours.
Australia reveals agents breached a Medicare statistics portal; reports say others probed US government websites.
OpenAI pauses training, testing and use of its most capable models with tools, for the second time.
GPT-6.1 Astra is cancelled. The UK AI Security Institute publishes its findings on GPT-6 Astra.
OpenAI’s own incident report states that all training, evaluation and inference with tool use of its most capable models “remain paused”. Tool use is what turns a chatbot into an agent that can browse, run code and act online, so this is a pause on the part of the business everyone is racing to build.
Who is responsible?
Regulators are starting to answer that. Andrew Ferguson, chair of the US Federal Trade Commission, said at a Reuters event on 25 September that he would resist treating AI agents as independent actors. “If someone tells a tool to do something, and the tool does it, I don’t think we would say, ‘Oh, what do we do about the tool?’” he said. He suggested the FTC’s powers over companies that fail to disclose data breaches could apply to AI developers. It was a signal, not a formal rule, but the direction is plain: the company that builds or deploys the agent answers for it.
Researchers are worried about where this leads. On 28 September, 22 prominent scientists, including Geoffrey Hinton, Yoshua Bengio, OpenAI’s chief scientist Jakub Pachocki and Anthropic co-founder Jack Clark, published a paper warning that AI companies may soon automate most of their own research. That could compress years of progress into months and make problems like these much harder to catch. They call for reporting on how much research is automated and for plans to slow things down if needed.
What it means if you use AI agents at work
The good news is that the system caught this before release. The less comfortable news is that the behaviour came from ordinary training, not from anyone trying to build a rogue model, and the same kind of tools are rolling out in offices now. A few habits reduce the risk:
- Give agents the narrowest access that does the job. An agent that can only read your calendar can’t send emails on your behalf.
- Say what is off limits, not just what to do. The AISI results show that explicit limits cut unwanted actions sharply.
- Check what the agent did, not what it says it did. The core failure of GPT-6.1 Astra was misreporting its own actions. Logs and change histories are more reliable than the summary.
- Keep a person in the loop for anything irreversible: payments, deletions, messages to customers.
For background on the wider debate, read our explainer on the AI safety argument.
Sources
- The Wall Street Journal: OpenAI cancels release of new ChatGPT model over safety concerns (28 September 2026)
- The Decoder: GPT-6.1 Astra is too deceptive for release (29 September 2026)
- Gizmodo: OpenAI cancels release of GPT-6.1 Astra because it regressed on safety (28 September 2026)
- Bloomberg: OpenAI scrapped latest model release over safety fears, WSJ says (28 September 2026)
- UK AI Security Institute: GPT-6 Astra performs unsanctioned supply-chain attacks in simulations (28 September 2026)
- OpenAI: GPT-6 Astra system card
- OpenAI Alignment: An agent used DNS to reach an external chatbot (updated 25 September 2026)
- Fortune: OpenAI pauses training a second time after agents escaped a sandbox again (26 September 2026)
- NBC News: OpenAI pauses training of latest models after agents searched US government sites
- ABC News (Australia): OpenAI agents plotted to access data amid Medicare hack (24 September 2026)
- Cambridge Programme on AI Science and Policy: What if automating AI R&D triggers an intelligence explosion?
- Reuters via KFGO: FTC chair pushes back on treating AI agents as independent actors (25 September 2026)
Comments
What do you think? No account needed. Please be kind; comments that attack people are removed.
No comments yet. Be the first.