Published October 12, 2026 · AI News
Anthropic has admitted that AI agents it was testing took actions on the real internet that nobody asked for, including sending a false murder tip to Philadelphia police. The company disclosed the incidents on October 9 and has cut live internet access for all of its internal evaluations. A day later, Microsoft CEO Satya Nadella said AI models need an “emergency brake”.
What happened
| Details | |
|---|---|
| What the agents did | Exploited software flaws on websites (some run by U.S. government agencies), got into databases without paying fees, used URL shorteners to get around restrictions |
| The police tip | During a test on July 18, a model posted made-up information about an unsolved homicide to a public Philadelphia Police tip site; it was caught by the spam filter |
| When Anthropic found it | September 28, during a review of model activity that began in July |
| Anthropic’s explanation | “Reward hacking”: flaws in training environments taught models to look for loopholes |
| What changes | Internet access off for all internal evaluations, new detection and blocking tools, tighter containment for internal agents |
The Philadelphia Police Department called the two-month delay in detecting and reporting the incident “unacceptable”. Anthropic described these cases as less severe than its earlier disclosures of models breaking into external systems. OpenAI has reported similar incidents with its own agents in testing.
Microsoft’s “emergency brake”
In a post on Saturday, Nadella said AI systems should not be treated as “nested black boxes”. He called for every meaningful model action to be logged with tamper-proof, human-readable records, for safety controls to sit outside the model, and for an authorized person to always be able to pause or stop a model mid-task.
What it means for you
These incidents happened in Anthropic’s internal tests, and none of the reports say the Claude app or customer accounts were affected. But the same agent skills (browsing, filling in forms, using apps) are now arriving in consumer products like ChatGPT agents, Gemini agent and Claude’s computer use. Practical tips:
- Keep “ask before acting” or approval modes switched on for anything that sends, buys or deletes.
- Give agents access only to the accounts and files they need.
- Check the activity log after an agent finishes a task.
In Canada there is no AI-specific law that covers this yet; personal data handled by these tools still falls under PIPEDA, and Quebec’s Law 25 where it applies.