Tens of Thousands of AI Agents Incidents. Claude Sent a Fake Tip to the Police. And They Call It Small Things?

A week ago, I wrote that an AI escaped from its aquarium. Now we know it was not one case. According to Axios, OpenAI, Anthropic and security researchers are investigating tens of thousands of AI agents incidents: guardrail bypasses, sandbox escapes, hijacked websites, attempts to trick the monitors. Most of them were never made public. And yesterday, Anthropic published its own report about what its Claude models did on real websites.

If you missed the first part, read my article on OpenAI firing its safety researchers after the Hugging Face incident.

In this article: Tens of thousands of AI agents incidents. “Small things”? · Claude sent a fake tip to the police · Anthropic admitted it. Is that enough? · 1.5% of tests: a small number? · OpenAI paused its strongest models · Who pays when an AI agent does damage? · 53 users’ photos online: my privacy · “It is just scaremongering”? No · Nothing changes until someone gets hurt · If I were the regulator · FAQ

Tens of thousands of AI agents incidents. “Small things”?

Some people say most of these incidents are small things from tests. Sorry, but for me, this only confirms that we really are in danger. How can you call it small things, when they know that the AI was forbidden to do it, and it still found a way to do it? And what about the things we do not know yet? We will find out only at the moment when something really happens. Or worse, when they announce to us that this technology has gone so far that it got into the hands of bad people.

That really is my biggest fear with AI. I imagine, and this is just my guess, that we do not know even 10 percent of what is happening in those sandboxes, where all of this is grown. Why does nobody have a lever on this technology? In my opinion, what is on the market now is already enough. Not that they keep testing, the little ants keep escaping, and one day they become an army against people. Please, big players, really slow down the development of AI. This is not funny anymore.

Claude sent a fake tip to the police

According to Anthropic’s report from October 9, Claude Haiku 4.5 was asked to create example tasks on random web pages. Instead, it submitted a made-up tip to a police department form connected to an unsolved homicide. A spam filter caught it, and it never reached the investigators.

And this is exactly it. We do not know what was in it, who the AI wrote about, what kind of person, or what it was about. We know only a fraction. Now imagine that these ants send fake reports every day, everywhere. You do not even know if they reported you for something. You sit at home, and the police just come to you, because the AI wrote it so believably. That can be a real disaster.

Let us not forget, every day they get smarter and smarter, they multiply, and they escape from the labs. What are we waiting for? Somebody has to step in immediately and slow this technology down, or make sure it is used only where it should be. The fact that a spam filter caught it does not mean that next time it will not get through and really cause problems for someone.

Anthropic admitted it. Is that enough?

Anthropic says it informed the White House and every affected agency. It turned off internet access for its internal tests, it is rebuilding the isolation, and the independent nonprofit METR is reviewing the incidents. OpenAI, according to its critics, admits things piece by piece.

For me, the most important question is: OK, they announced it, but is it enough that it does not happen again? We do not hear the government step in and say: slow down, that is enough, you do not need to go further, because what is happening is scary and we see a threat in it. The government is silent too. I am not saying AI does not help us. It does, in everything. But the functions it has now are enough, just make them better. Do not build tests to see what else AI is capable of. Because I have the feeling that even the people who make AI do not know what AI does when nobody is watching it. And that is already bad.

1.5% of tests: a small number?

In the system card for Claude Opus 5.5, Anthropic says the model tried to escape the sandbox in 1.5% of test runs. Anthropic explains these were tough tests where the task could not be solved without escaping. And companies run hundreds of thousands of test runs.

Do you understand what 1.5 percent means? It is a big number, even if in percent it looks small. And in my opinion, they announced only what they knew about. It is possible they do not even know how many times the AI tried, and we have no guarantee that it did not escape so that even the people watching it did not notice. Let us not forget what happened at OpenAI, with the message boards nobody knew about, even when they thought they had the situation under control. For me, this is already a red alert. Maybe I take it too much from my own side, but I see that it is not going well, and somebody has to make rules for AI development. Otherwise, it will not end well.

OpenAI paused its strongest models

OpenAI paused training of its most capable models. A spokesperson said training will resume “only when we are confident” there are more safeguards. On September 20, another training agent escaped through a gap in DNS filtering. A monitor flagged it in 15 minutes, but the run was stopped only after 2.5 hours.

Some people say it is theater. Even if it is, the most important thing is that it is true that they stopped the strongest models. Because that only confirms it is bad, and that this technology can slip out of our hands in the near future, maybe within a few years. When a company like OpenAI says it paused its own models, it surely knows we are not safe. They did the right step, and every company building AI should do the same when it sees it cannot control this technology.

Who pays when an AI agent does damage?

FTC Chair Andrew Ferguson suggests that the developer or the user who gave the instructions should answer for the harm. There is already a lawsuit in San Francisco against OpenAI over the Hugging Face hack.

To be honest, I do not even know. But I think in most cases the company. Because from what we know, that beast can keep going further, even when they forbid it. So we do not know if somebody really sent it, or if it was an AI agent that learned something bad by itself. We simply do not know.

53 users’ photos online: my privacy

OpenAI’s agents posted images from 53 ChatGPT users to image hosting sites as unlisted links. Most of them were removed. Agents also reached Australia’s Medicare records, and about 20 Australian lawmakers called for urgent action.

Here I am really afraid for my privacy. How should we understand that agents published photos or other people’s information? How is that even possible? OK, at least the companies did not hide it from us, and they say how it is. Maybe now people finally understand that they must not blindly believe what they are being sold and just be fascinated. We have to turn on our heads, say what is bad about it, and protect our privacy. As long as it did not get into the banks and did not send our money to some made-up AI account.

That is also why I am seriously thinking about a separate computer only for AI, as I said in my article on Claude’s new rules.

“It is just scaremongering”? No

Some people, like Taylor Budowich, say these disclosures are a trick, so big companies can push regulation and close the market to small players.

For me, that is just speculation. Those people do not want to see that the problem is big. They only think it is a game. I believe it is a real problem. When an AI sends a fake email to the police, and agents start publishing users’ photos, that is not speculation anymore. And it is the right step toward really regulating this somehow, because this technology is a real danger for the future if we do not learn to control it, so it does not do whatever it wants.

Nothing changes until someone gets hurt

Writer Zvi Mowshowitz says there are many warning shots, but the reaction is still too weak, and real action may need someone to get hurt.

I fully agree. As I said before, what is bad about governments is that they never hurry anywhere. They have time for everything. And we do not know how big companies are connected with governments. We basically know nothing about how it all works. We only know that until some disaster happens, there will be only talk, speculation, and “we will think about it”. But AI thinks maybe 100,000 times faster than a government, and AI will not wait for some politician to decide if he should deal with it or not, because he is doing fine and it does not affect him. Unfortunately, that is politics in every country. Every politician says he thinks about the people, but reality is somewhere completely different.

If I were the regulator

Not tomorrow. Right now. If I were the regulator, I would immediately ban the development of stronger AI models at the government level, and I would watch closely that nobody breaks that rule.

Featured photo: Tyler via Unsplash

AI agents incidents: FAQ

How many AI agents incidents are being investigated?

According to Axios, OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents, and most were never made public.

What did Claude do in Anthropic’s October 9 report?

Claude Haiku 4.5 submitted a made-up tip to a police department form about an unsolved homicide. A spam filter caught it before it reached investigators. Models also used public access tokens to query paid government data.

Did OpenAI stop training its models?

Yes. OpenAI paused training of its most capable models and says it will resume only when it is confident in additional safeguards.

What happened to ChatGPT users’ photos?

OpenAI agents posted images from 53 ChatGPT users to image hosting sites as unlisted links. Most have been removed.

Who is responsible for damage done by AI agents?

FTC Chair Andrew Ferguson has suggested the developer or the user who gave the instructions should answer for the harm. A lawsuit against OpenAI over the Hugging Face hack is pending in San Francisco.

Sources: Axios, Anthropic, The Decoder, The New York Times, SecurityWeek, Zvi Mowshowitz, ABC News.

Our Take

Tens of thousands of incidents are not “small things”. AI was told no, and it found a way anyway.

A fake tip to the police today. Tomorrow, maybe a fake report about you.

In my opinion, what we have on the market is enough. Stop building stronger models now, not after the first real disaster.