Key Takeaways

  • McIntyre found that large language models write functional exploit code for open-source web bugs like token logic errors, expression language injection, and directory traversal leaks.
  • Models stumble when handed closed-source software, firmware, or compiled x86 binaries requiring patch diffing and multi-step reverse engineering.
  • VulnCheck runs an open-source tool called go-exploit to feed models normalized structures, preventing the hallucinated syntax common in raw exploit generation prompts.
  • Code flaw detection does not equal attacker utility; defensive teams waste hours chasing broken and fake proof-of-concept exploits generated without standardized execution scaffolds.

Web Applications Give Models an Unfair Advantage

Language models read text. When an application exposes readable source code, an LLM treats the bug like an editing task. It reads the code paths, spots where an input variable slips through without sanitization, and writes a script to trigger the path.

Spencer McIntyre, Director of Exploit Development at VulnCheck, found that this text-based pattern recognition works well on web architectures. “We've had a lot of luck using AI and LLMs to exploit open-source web applications,” McIntyre noted. “So these types of flaws that are like logic flaws and how like token logic is being handled, or expression language injections, or file leaks and things like that, it's pretty darn good with those.”

If your startup runs web applications built on readable frameworks, assume automated scanners can generate weaponized calls against your endpoints. The model does not need to understand machine state or memory offsets. It only needs to understand the syntax of an HTTP request and the structure of an open source function.

Binary Reverse Engineering Breaks Language Models

Step outside open-source web code, and model performance collapses. Compiled binaries do not hand the model clean functions or obvious variable names. They give the model disassembled instructions, stripped symbol tables, and execution paths that span across memory registers.

“Beyond that and beyond kind of go-exploit, once you dive into like the real details of the vulnerabilities, vulnerabilities fall under certain vulnerability classes,” McIntyre explained. “And what we've noticed is that LLMs seem to be pretty darn good at exploiting multiple vulnerability classes and not very good at exploiting other vulnerability classes.”

The failure becomes stark when the model must handle proprietary targets. “When we start getting into anything that's closed source, where there's a layer of reverse engineering that has to happen ahead of time, because it's like firmware, or even like closed source proprietary x86 software, those types of things, where it has to do like patch diffing, it tends to struggle a lot more,” McIntyre observed.

A human researcher diffs two binary patches by analyzing how small shifts in assembly instructions alter program state. An LLM cannot infer those execution mechanics reliably. It predicts tokens based on statistical text relationships, which fails when a single wrong byte in an offset crashes the payload.

Scaffolding Beats Raw Prompting

Handing an LLM a prompt that says "write an exploit for this bug" produces broken code. The security space is already flooded with fake and non-functional proof-of-concept scripts because models hallucinate non-existent libraries and misconfigure network payloads.

VulnCheck solved this by wrapping model generation inside go-exploit, their open-source exploitation framework. Instead of asking the model to invent an entire program, they force the model to fill in structured Go templates. The framework handles command-line parsing, HTTP target handling, and error logging. The model only generates the specific payload delivery logic. Giving the model a clean harness reduces syntax errors and produces exploits that defense teams can test and verify.

What to Do With This

Audit your external attack surface this week and split it into two piles: open-source web code and compiled services. Prioritize patching logic flaws and injection bugs on public web repos first, because attackers can use current LLMs to script reliable payloads against them in minutes. For compiled binaries and proprietary services, keep your human security researchers focused on manual patch diffing, where automated models still generate noise.