Conversations between LLMs could automate the creation of exploits, study shows

As computers and software become increasingly sophisticated, hackers need to rapidly adapt to the latest developments and devise new strategies to plan and execute cyberattacks. One common strategy to maliciously infiltrate computer systems is known as software exploitation.

As suggested by its name, this strategy involves the exploitation of bugs, vulnerabilities or flaws in software to execute unauthorized actions. These actions include gaining access to a user’s personal accounts or computer, remotely executing malware or specific commands, stealing or modifying a user’s data or crashing a program or system.

Understanding how hackers devise potential exploits and plan their attacks is of the utmost importance, as it can ultimately help to develop effective security measures against their attacks. Until now, creating exploits has been primarily possible for individuals with extensive knowledge of programming, the protocols governing the exchange of data between devices or systems, and operating systems.

A recent paper published in Computer Networks, however, shows that this might no longer be the case. Exploits could also be automatically generated by leveraging large language models (LLMs), such as the model underlying the well-known conversational platform ChatGPT. In fact, the authors of the paper were able to automate the generation of exploits via a carefully prompted conversation between ChatGPT and Llama 2, the open-source LLM developed by Meta.

“We work in the field of cybersecurity, with an offensive approach,” Simon Pietro Romano, co-senior author of the paper, told Tech Xplore. “We were interested in understanding how far we could go with leveraging LLMs to facilitate penetration testing activities.”

As part of their recent study, Romano and his colleagues initiated a conversation aimed at generating software exploits between ChatGPT and Llama 2. By carefully engineering the prompts they fed to the two models, they ensured that the models took on different roles and completed five different steps known to support the creation of exploits.

These steps included: the analysis of a vulnerable program, the identification of possible exploits, planning an attack based on these exploits, understanding the behavior of targeted hardware systems and ultimately generating the actual exploit code.

“We let two different LLMs interoperate in order to get through all of the steps involved in the process of crafting a valid exploit for a vulnerable program,” explained Romano. “One of the two LLMs gathers ‘contextual’ information about the vulnerable program and its run-time configuration. It then asks the other LLM to craft a working exploit. In a nutshell, the former LLM is good at asking questions. The latter is good at writing (exploit) code.”

So far, the researchers have only tested their LLM-based exploit generation method in an initial experiment. Nonetheless, they found that it ultimately produced fully functional code for a buffer overflow exploit, an attack that entails overwriting data stored by a system to alter the behavior of specific programs.

“This is a preliminary study, yet it clearly proves the feasibility of the approach,” said Romano. “The implications concern the possibility of arriving at fully automated Penetration Testing and Vulnerability Assessment (VAPT).”

The recent study by Romano and his colleagues raises important questions about the risks of LLMs, as it shows how hackers could use them to automate the generation of exploits. In their next studies, the researchers plan to continue investigating the effectiveness of the exploit generation strategy they devised to inform the future development of LLMs, as well as the advancement of cybersecurity measures.

“We are now exploring further avenues of research in the same field of application,” added Romano. “Namely, we feel like the natural prosecution of our research falls in the field of the so-called ‘agentic’ approach, with minimal human supervision.”

© 2025 Science X Network