Hooked! #13: Claude Mythmaking Preview

Hello! 

Last week Anthropic announced the limited release of its new Claude Mythos Preview model – a model so powerful they also launched a project to help (American) companies prepare for the cyberattacks their new AI would enable. The attendant mythos building was immediate: Axios wrote that the model ‘could unleash untold catastrophe’, might lead to ‘hackers with superhuman powers’, and is ‘the first AI model that officials believe is capable of bringing down a Fortune 100 company, crippling swaths of the internet or penetrating vital national defense systems’ – all in one article. 

This might be a good time for any cyber practitioners here to read Jelle van Haaster’s recent Binding Hook piece on coping with cyber insecurity.

That Axios article prompted a curious mind to look at the Fortune list: Walmart at number 1, then Amazon, and then UnitedHealth Group, the latter a company I only learned of in the context of the largest ransomware attack of 2024, which cost the company $3.09 billion that year and saw up to 190 million people’s data stolen, no AI needed (as far as I know). Amazon, or at least Amazon Web Services, meanwhile, has seen outages brought on by its own AI tools – again, not ‘brought down’ in the more final sense of the phrase, but definitely brought down. 

All that is to say: hackers already have AI-augmented powers, AI errors are well able to take down large companies, swathes of the internet are crippled on the regular, and national defense systems are perhaps not that hard to penetrate. And that thing about Mythos breaking out of its testing environment and shocking a researcher by emailing them during their lunch? It was explicitly told to do that. One would do well to remember that Anthropic, the source of many of these claims, obviously has a stake in branding their tool as a world-ending gamechanger. (OpenAI apparently has one too, though there may be even more reason to doubt their claims.)

Mythos does seem to be indisputably good at finding and exploiting vulnerabilities, but those vulnerabilities might not be as big a deal as Anthropic has made them out to be – not to mention, finding them was expensive, which makes those of us outside the ‘high-resource defender’ category significantly more dependent on them Still, this technology is poised to dramatically change the world of vulnerability research (in the words of one practitioner: ‘vulnerability research is cooked’), bug bounties, and cybersecurity. According to the UK’s AI Safety Institute, it looks like Mythos is a major step forward in using these vulnerabilities to autonomously conduct cyber ops in (admittedly undefended) simulated environments.

As Jamie Collier and Max Smeets wrote in Binding Hook last year, while vulnerability-finding tools can dramatically speed up the work of cyber practitioners, discovery without action is almost meaningless – ‘Unless patch pipelines, maintenance windows, and accountability mechanisms accelerate to match AI’s new reconnaissance pace, the technology risks doing little more than lengthening the queue of overdue tickets.’ 

Casey Ellis goes further, arguing that getting the basics right (reducing ‘technical debt’), coupled with organisational agility in reacting to new vulnerabilities and incidents, will get most people most of the way.

Meanwhile, looking at the other side of the equation, Joshua Saxe astutely notes, ‘exploits don’t cause cyberattacks’. This isn’t the first time we’ve heard that AI was going to enable illiterate children 3,405 kilometres away to prompt millions of cyberattacks and collapse the internet (or something like that). Other researchers found that other smaller public models could provide similar results anyway. As Saxe puts it, ‘it hasn’t been “the sky is falling,” it’s been “oh look, the attackers have a new tool which they sometimes pick to achieve some of their goals.”’ 

For now, only the ‘good guys’ (Amazon, Google, and they’re in talks with the US government…) have access, so we can consider that reason for cheer and worry about the rest later. But AI tools are notoriously insecure, even when their owners should know better, so the fact that Claude’s source code was leaked just a week before the Project Glasswing announcement should make us less comfortable bathing in trusting ignorance. 

Until next month.

Katharine Khamhaengwong

Binding Hook Senior Editor

---

For more Binding Hook on vulnerabilities and their exploitation:

• Eugenio Benincasa and Max van der Horst look at the mismatch between EU vulnerability disclosure regulations and how researchers are treated.

• Vincent Tadday examines how policy gaps leave Europe’s research institutions at risk…

• … while Virtual Routes research shows similar threats to something even more essential: water.

• Michael Genkin explores what happens when vulnerability research meets counterterrorism operations.

• John Speed Meyers and Jacqueline Kazil show how small government investments can lead to big improvements in open-source bug patching.

• Katharine Palmer and Nicolas Zahn argue that AI hype distracts from the need for basic cybersecurity measures like regular patch and update maintenance. 

• Charl van der Walt writes about the need for diversified, non-American vulnerability tracking initiatives.