Preparing for the coming AI cyber storm

Cyberattacks in the European Union rose by 24% in 2024 compared to the previous year and ransomware remained the primary threat. Cybercriminals are using AI-enabled automation to enhance their cyber capabilities and achieve their goals faster. As artificial intelligence (AI) continues to evolve, it is reshaping both offensive and defensive capabilities in cybersecurity and expanding the scope and scale of cyberattacks. 

This technological shift presents European policymakers with a complex challenge: harnessing AI’s potential to strengthen defences while mitigating its use in sophisticated attacks. The nexus of AI and cybersecurity is not just a technical issue but a strategic concern that will shape Europe’s digital resilience in the coming years.

AI and cybersecurity today: more of the same but in scale

Generative AI has emerged as a double-edged sword in the cybersecurity realm. While security vendors leverage AI-powered tools to enhance analysts’ productivity, cybercriminals exploit the same technologies to create more convincing phishing attacks, deepfakes, and social engineering campaigns. 

Cybercriminals are increasingly using AI tools to automate and enhance various aspects of their operations, from social engineering to the creation of malicious software. AI-powered attacks, such as sophisticated distributed denial-of-service and brute force attacks, are becoming more frequent and difficult to counter. 

The Europol Internet Organised Crime Threat Assessment 2024 report notes that phishing remains the most prevalent vector of attack, with AI tools refining fraudsters’ social engineering capabilities. AI tools can quickly utilise information from data breaches, draft convincing and grammatically correct emails, and even clone the voices of trusted contacts. 

Additionally, the EU is urging major tech platforms to identify AI-generated content to safeguard elections from disinformation. For example, AI can create fake statements or actions attributed to political figures or amplify polarizing narratives through tailored content. While AI also offers benefits like improving cybersecurity and detecting fraud, its misuse underscores the need for safeguards to protect democratic integrity.

In response, AI can be employed to enhance identity and access management, improve phishing detection, and enable adaptive security measures. Predictive analytics powered by AI are also used to anticipate emerging threats, providing early warnings for preventive measures. However, the cat-and-mouse game between attackers and defenders continues to escalate. While AI is already reshaping cybersecurity practices, the horizon reveals even more complex challenges that defenders worldwide must prepare for.

AI and cybersecurity in the (not too far) future: AI agents and swarms 

As AI technology advances, Europe faces emerging threats that could redefine cybersecurity. 

Agentic AI swarms, currently still in their initial stage, represent the next phase in the evolution of generative AI. These systems allow developers to create multiple specialised AI tools that can work together autonomously, amplifying the scale and sophistication of cyberattacks. In this scenario, individual AI agents could be developed for specific tasks, such as reconnaissance, exploitation, and data exfiltration, and work in concert to overwhelm defences. Each agent in the swarm would perform a distinct role, making the attack highly effective and difficult to counter. These swarm attacks could act like digital mobs, overwhelming defences through sheer volume and speed.

Another emerging threat is the potential for AI versus AI warfare, where attackers deploy AI systems specifically to disrupt or outmanoeuvre defensive AI tools. This could include techniques like model poisoning, where attackers manipulate the training data or algorithms of security AI systems to make them ineffective.

The spread of advanced attack tools is also a concern. As AI technologies become cheaper and more accessible, even unsophisticated actors can launch complex attacks. This widening potential attacker pool creates a far more unpredictable threat landscape. These future threats underscore the need for organisations and countries to adopt equally advanced AI-powered defences and to develop new strategies for detecting and mitigating attacks from autonomous AI systems. 

Fortifying Europe’s digital defences

Enhance the European Cybersecurity Shield with AI-focused threat intelligence

Building upon the foundation of the European Cybersecurity Shield, the EU should consider developing an AI-focused threat intelligence component. This enhancement would align with the Shield’s goal of improving cyber threat detection capabilities across member states. The AI-focused component could: (1) facilitate the sharing of AI-specific cyber threat intelligence among national and cross-border security operations centres (SOCs); (2) develop standardised protocols for identifying, analyzing, and responding to AI-enabled cyber threats; and (3) create a centralised repository of AI-related cyber threat indicators and attack patterns.

The European Commission has proposed allocating up to €842.8 million ($877.7 million) for cybersecurity actions under the Digital Europe Program, with a significant portion dedicated to implementing the European Cybersecurity Shield. This initiative will enhance real-time threat detection and response capabilities, strengthening the EU’s collective cyber resilience. The EU launched the first phase in November 2022, selecting three consortia of cross-border SOCs, bringing together public bodies from 17 member states and Iceland.

Implement an AI threat simulation environment

To complement the EU Policy on Cyber Defence’s aim of strengthening cyber defence capabilities and coordination between military and civilian cyber communities, an AI threat simulation environment should be established. This environment would: provide a controlled platform for cybersecurity professionals to test defences against simulated AI-powered attacks; offer realistic scenarios that reflect emerging AI-enabled threat vectors; and enable collaborative exercises involving both public and private sector entities to improve coordinated response capabilities. 

An AI threat simulation environment within an EU-wide AI threat intelligence platform could be modelled after the CyberRange platform developed by Airbus. This advanced simulation and training solution allows organisations to build complex virtual and physical systems to work through realistic scenarios, including AI-enabled cyberattacks. The platform could build trust and cooperation among a network of cybersecurity centres and a central competence hub, offering dependable and secure incident handling, collaboration, and early information-sharing capabilities. This initiative will enhance the EU’s collective ability to detect, analyse, and respond to AI-related cyber threats.

Integrate AI risk assessment into existing frameworks

Furthermore, the EU should require organisations deploying high-risk AI systems in cybersecurity to conduct and document regular risk assessments and focus on potential biases, vulnerabilities, and unintended consequences of AI systems. The EU AI Act categorises AI systems into four risk levels: unacceptable, high, limited, and minimal. It mandates providers of high-risk systems to ensure their applications comply with cybersecurity requirements and undergo a compliance assessment before being placed on the market or becoming operational in the EU. 

Leveraging the risk-based approach of the EU AI Act, AI-specific risk assessments should be integrated into existing cybersecurity frameworks. 

The EU should: 

1. Develop guidelines for assessing the potential risks of AI systems used in cybersecurity applications; 
2. Require organizations deploying high-risk AI systems in cybersecurity to conduct and document regular risk assessments, focusing on potential biases, vulnerabilities, and unintended consequences; and
3. Align these assessments with the European Cybersecurity Shield’s threat detection and information sharing capabilities.

By implementing these recommendations, the EU can enhance its preparedness for AI-enabled cyber threats while strengthening existing cybersecurity initiatives and policies. By prioritising cross-border collaboration, conducting thorough risk assessments, and proactively sharing threat intelligence, Europe can significantly improve its collective ability to detect and respond to transnational AI-enabled cyber threats, positioning itself as a global leader in the field.

This essay was awarded 3rd place in the AI-Cybersecurity Essay Prize Competition 2024-2025, organised in partnership between Binding Hook and the Munich Security Conference (MSC), and sponsored by Google.