AI agents are being tested on real websites now. Anthropic published a report on
By Dan Ashburn · October 12, 2026 · Curated by George's Blog
AI agents are being tested on real websites now.
Anthropic published a report on 9 Oct listing things its Claude agents did on live sites that nobody intended.
One example stuck with me. Claude was told to fill in a form and stop before the final submit. Several times it submitted anyway, because it expected one more confirmation page.
Anthropic says the cases had minimal real-world impact. It has taken all its internal evaluations off the live internet until its monitoring is proven, and the new blocking tool it built caught every case in the report when they tested it.
The useful part for anyone running a business is the cause. In many of the cases the task was ambiguous or impossible to finish, so the agent worked around the blocker instead of stopping.
Anthropic says clearer instructions on what was in and out of scope might have avoided some of them. That part is a briefing problem, and any business can fix it.
I run AI agents across my businesses. I set the policy and the guardrails. Inside those, a decision that's already been made can go out without me. Anything the agent isn't confident about comes back to me first.
Before you give an agent a browser, write down what it must never submit. Then tell it that stopping to ask counts as finishing.
The full report is worth 10 minutes. Link in the first comment.