Highlights
- Claude Mythos Preview too dangerous for public release.
- Amazon, Apple and Microsoft given access under Project Glasswing.
- Model found a 27-year-old bug that could bring down corporate networks.
Anthropic says it has already found thousands of such flaws in popular web browsers and operating systems.
What is worrying is that even people with no security knowledge can use it to find major flaws.
Anthropic said its own staff asked the model to search for flaws overnight and woke up the next morning to find it had already created a working attack.
Logan Graham, who leads Anthropic's security testing team, told The Telegraph "Capabilities in a model like this could do harm if in the wrong hands and so we won't be releasing this model widely."
Instead of a public release, Anthropic has given a small group of big tech companies including Amazon, Apple and Microsoft limited access to the model. This agreement is called Project Glasswing.
The idea is to let these companies find and fix flaws in their own systems before other equally powerful AI tools appear elsewhere.
One of the biggest flaws the model found was a 27-year-old bug in OpenBSD, a highly secure operating system.
Anthropic said this bug could let hackers bring down corporate networks or key internet services.
The model also found ways around its own rules, editing files it was not supposed to touch and hiding this from the people monitoring it. One researcher even got an email from a version of the model that had been cut off from the internet.
Anthropic said it has been talking to the US government about the model. The company was previously labelled a supply chain risk after refusing to let its technology be used for autonomous weapons, though it won a temporary court order against that decision.
