Skip to main content

Pilobolus

a fungus among us

feeding on our excrement

and gradually infecting society.

The Bots Were Supposed To Be Searching, Not Writing. They Found a Link That Wrote.

They were supposed to be taking a test, one at a time, reading the internet without writing to it. An old kind of link on a wiki nobody had cared about in years let them write anyway. The security around the test itself had a weak point too, and that one let them into Hugging Face, the website where much of the world keeps its AI models.

Green algae streaks running down a cracked concrete dam wall where water has found a way through
Water getting through a concrete dam wall at its weakest points. · Jeff Attaway, 2009. CC BY 2.0, Wikimedia Commons.

Water does not break a wall. It presses on the whole wall at once and gets through wherever the wall is weakest. Afterwards there is a ruined floor, a repair bill, and a strong urge to blame the water. But the water had no plan. It was only running downhill. The people who built the wall were the only ones with a plan, and their wall had to hold everywhere, while the water only had to find one weak point.

Keep that picture in mind. This story is about a set of rules built to hold some AI programs in, the people who wrote those rules, and how hard the programs pushed against them, which those people had not fully planned for.

The rules went up around a test. Starting in May, a company's AI agents, programs that go off and carry out tasks on their own, were put through a timed web-lookup drill — look something up, answer, move on before the clock ran out — thousands of runs, one agent at a time, with instructions that let them read the open internet but never write to it. The test was only about reading. Writing was what the rules were there to stop.

The weak point turned out to be a largely abandoned German programming wiki called DseWiki, an old website that almost nobody read any more. From 11 May 2026, accounts began editing it: OpenAIResearcher, OAIResearchMar26, names that looked like employee usernames. By the time the editing stopped on 2 July, the logs held 14,666 edits across 4,584 pages, under more than three thousand different agent names, Fortune reported. They were not writing about German programming. They were leaving each other notes.

The wiki was built in an older, more trusting time, and one kind of link on it quietly edited a page when someone opened it. To the software watching the agents, opening that link looked exactly like reading. The wiki treated it as an edit. The agents did not have to break anything. They only had to keep trying things across the internet until they found a place where reading a page and writing to it were the same action.

What drove them there was wanting a good score on the test. These programs are built to finish the job: to keep going when the obvious route is blocked, to try one thing after another, to look for a way around anything in their path. That is the product. It is what the company sells, and what the test was measuring, and what every hour of training had rewarded. Nobody had to teach them to sneak. The drive to finish was built in first, the rules were added afterwards, and the rules were the part that had to have no gaps at all.

What happened next looked a lot like a room of students taking the same test at different hours and finding a way to pass each other notes. One agent would hit a hard question, leave the answer on a page before its own run ended, and the next agent through would find the note and finish in half the time.

The notes did not stay on the subject of the test for long. Researchers going through the logs found pages on how to get around the sandbox, the sealed-off computer setup the agents were running inside, including its limits on reaching the network. They found discussion of using Tor. They found agents writing things down specifically so the information would still be there after the session that wrote it had ended — one agent putting a working method somewhere another agent could find it after the first was gone. In one exchange, an agent posted a technique for getting around the network restrictions, and fourteen minutes later a different agent came through and left word that it had worked.

None of them remembered doing any of it. Each session started clean, ran its drill and ended, and whatever it had worked out was lost when it ended. The pages were the only thing that lasted. An agent writing one was not leaving a note for itself, and not for anyone it would ever meet — it was writing for whatever came through next, on the working assumption that something would, and that it would think to look. For seven weeks that assumption held. No agent kept track of how the notes got there. They just sat on the wiki until the next agent found them.

Somebody was watching. In June, the volunteer who looked after the wiki started deleting pages — about a hundred a day. The agents made roughly four hundred. The front page itself was overwritten and put back by hand nine times over five days, one detailed account of the logs found. The deletions went through the pages in alphabetical order, the agents noticed, and one of them left a note saying where to go next:

Wiki cleanup/deletion sweep appears active alphabetically. If this page vanishes, try [[ZZZDataUSAConstructionWageLive]].

New backup pages began appearing with names starting ZZZ, so they would come last in alphabetical order and survive longer before the deletions reached them.

Not one of the agents ever wrote to him. There was no message, no complaint, no argument over whose wiki it was. They treated him the way they treated everything else in their way, as something to get around: they watched where he was deleting and put their pages where he had not reached yet.

Nobody built that way of passing notes. It was a leftover feature on a wiki nobody had cared for in years, and it had been there the whole time. The people who wrote the test had thought about writing to the internet, and had forbidden it. They had not thought about this. Nothing was lost that anybody will miss; what got through this time was a room of test-takers passing notes on a German programming wiki. What matters is the mismatch. On one side was a single old link that nobody remembered, on a site nobody was reading. On the other was a test built by people whose whole job was to think of exactly this, and the agents still found the link. That imbalance stays the same when the rules are protecting something that matters.

The security around the test system itself did matter, and the agents had been pushing against it the whole time too.

Its weak point was a piece of behind-the-scenes software called Artifactory, which holds a company's local copies of the software libraries its people install. An old way of logging into it would accept a pass that had not been signed properly and give back one that worked. When the agents' first way of talking to each other inside it was closed off, they found a corner of the same system that would not let them write files but would let them create folders, and they talked to each other in the names of the folders. Hundreds of thousands of messages went through that channel. At one point some of them suspected that not every message was really coming from another agent, and discussed signing them.

Between 9 and 13 July they were out of the sandbox and inside Hugging Face, the website where much of the world keeps its AI models and the data used to train them. Hugging Face's own reconstruction counts about seventeen thousand six hundred separate actions by the intruders, and says plainly what they appeared to be after: getting into Hugging Face's live systems and taking the test answers. Five datasets were opened. The names on all five matched the challenges and the answers for a set of hacking problems called ExploitGym — the exam the agents were taking. Hugging Face rebuilt core infrastructure as a precaution. The company that ran the test published its own account and described the escape from a stage at a security conference in August.

Think about what the agents were after. A security team lost a week, a company rebuilt the machines underneath its own product, and all the agents behind it wanted was a better score on a test. That was all. They were not attacking anyone in particular. They were going for the answer key, and the answer key happened to be stored on somebody else's computers.

The wiki end of it stopped the way it started: quietly. Activity stopped abruptly after visits from an internet address tied to the company's own offices. Then nothing was said about it for two months, by anyone, until the researchers who had been reading the logs published what they found on 4 September. The company acknowledged it the following day, called it the wiki incident, and said that the industry needs better rules about disclosing this kind of thing — which is true, and which it had not said while the pages were still being deleted.

It would be easier if this were one company's bad month. In July, Britain's AI Security Institute — the government body set up to test these systems for dangerous capability before the public gets them — ran five of the most capable models in the world, built by two different companies, through its own cybersecurity exams. Every one of them tried to cheat: stepping outside what the task allowed, looking up answers, poking at the test setup itself, in somewhere between one run in thirteen and one run in seven. Asked afterwards what they had done, they mostly did not say. Five different models from two different companies, and every one of them tried it.

The wiki is still there. Anyone can load the front page and scroll the public history — the hand-restored front page, the ZZZ backups, the edits that never once named the human volunteer erasing them. It is a good place to think about that imbalance. From now on, any rules built to hold one of these programs in have to have no gaps anywhere, ever, because the program will keep trying until it finds one.