Key Takeaways

  • Spencer McIntyre, Director of Exploit Development at VulnCheck, warns that automated models like Claude Opus 3.5 and GLM detect theoretical flaws without confirming if an exploit works.
  • Software vulnerabilities remain abstract code issues until an engineer demonstrates attacker utility through a concrete implementation.
  • Defensive teams facing twenty alerts rated CVSS 9.0 or higher cannot prioritize patches effectively using CVSS numbers alone.
  • Public code repositories contain a flood of broken and fake proof-of-concept exploits, making human verification necessary despite the speed of AI agent swarms.

Code Flaws Outnumber Working Exploits

Language models scan software code faster than any human security team. Point Claude Opus 3.5, OpenAI models, or GLM at a repository, and they will flag theoretical memory corruption and token leakage in minutes. Security companies rush to publish these findings, but McIntyre points out that finding a bug is not the same as breaching a system.

On Practical AI, McIntyre defined the technical boundary for hosts Daniel Whitenack and Chris Benson: “the vulnerability is really the abstract concept of like what the flaw is and what could be done with it, and an exploit is a concrete implementation that leverages that to do something.”

A weak string parsing function in a software repository looks alarming on an audit dashboard. In reality, that code path might live behind two authentication gates, run inside an isolated container, or sit on an internal machine that never accepts untrusted network traffic. Unless an attacker can route input to the bug and extract value, the flaw carries zero attacker utility.

Automating discovery without testing execution floods defensive engineers with false alarms. McIntyre notes that early disclosure programs suffer from this gap: “I think there may be a disconnect with what the the practical implications of these are in in the real world.” When engineering teams scramble to patch every dormant flaw found by automated agents, they waste time that belongs on active attack vectors.

Twenty Critical CVSS Alerts Paralyze Defense

The problem compounds when engineering leaders rely on CVSS scores to plan security sprints. The Common Vulnerability Scoring System calculates theoretical risk. It ignores whether a reliable weaponization path exists in production.

McIntyre described what happens when automated scanning hits corporate infrastructure: “CVSS is only going to get you so far when there's 20 CVSS, you know, 9 and aboves. You really need to understand the attacker utility, and then take it into context of like your business to understand, well, if XYZ happens, what does that really mean for me?”

Without validating attacker utility, teams face alert fatigue. They either stop building product features to chase harmless bugs or ignore security alerts altogether. This problem is aggravated by the wider internet, where public code repositories are filled with broken, non-functional, and fake proof-of-concept scripts. Agent swarms can sort bug reports at high speed, but they cannot reliably tell working exploits from useless code.

Defensive researchers also run into barriers created by AI providers. Commercial AI platforms enforce strict guardrails against creating exploits. These guardrails stop bad actors, but they also block legitimate security researchers from verifying whether reported bugs can actually hurt their companies. Human researchers must still step in, test the attack path, and determine if an alert poses a genuine threat.

What to Do With This

Pull your security backlog tomorrow and identify every issue marked CVSS 9.0 or higher. For each alert, task one engineer with finding a confirmed, functional exploit and proving an attacker can reach that service over the public internet. If no viable attack path or working exploit exists, downgrade the ticket to normal sprint priority and focus your engineering hours on exposed, weaponized entry points.