Spencer McIntyre on Why Security Pros Are Quitting Frontier AI

Key Takeaways

• Formal verification programs run by proprietary AI vendors fail to unblock exploit development. VulnCheck director Spencer McIntyre notes that vetted researcher access rarely stops models from refusing basic vulnerability verification tasks.
• Offensive researchers using proprietary American models routinely downgrade to older model versions because safety filters on newer releases trigger immediate refusals on vulnerability code.
• Teams are moving sensitive exploit research to self-hosted open-weight architectures, specifically Chinese models like Qwen and GLM, to maintain autonomy over model responses.
• Security researchers are experimenting with weight abliteration, directly editing neural network weights to strip safety vectors and refusal mechanisms out of local models.
• Cloud API privacy concerns prevent defensive teams from sending unpublished zero-day flaws to commercial AI providers, making on-premise execution mandatory.

Commercial Safety Filters Block Legitimate Security Work

Spencer McIntyre directs exploit development at VulnCheck, where his team verifies whether discovered software bugs actually provide attacker utility. Doing that work with commercial frontier models has become an exercise in fighting system prompts. Proprietary vendors offer vetted cyber verification access programs, but McIntyre found that official access does not fix the friction: “The results aren't great. They don't really seem to change much of what we're able to do.”▶ 23:07

When defensive researchers feed vulnerable code or crash artifacts into frontier models, the safety layers assume hostile intent. The model treats exploit verification the same way it treats an active malware attack. Because frontier systems have tightened their refusal heuristics across recent updates, researchers have adopted an unexpected workaround: they downgrade. “The trends in the guardrails definitely seem to have gotten more strict,”▶ 23:42 McIntyre observed. “If you talk to anyone in offensive security research, basically the answer is, if you're using one of the American proprietary models, it's to downgrade it.”▶ 23:46 Researchers intentionally pick older, less restricted model checkpoints simply because older releases answer the prompt instead of lecturing the user.

Security Teams Are Migrating to Local Weights

Downgrading proprietary APIs is a temporary fix for a deeper operational problem. The real danger for a security team is data exposure. Handing an unpatched zero-day vulnerability to a hosted commercial API means passing private exploit primitives across someone else's infrastructure. McIntyre put the issue plainly: “From a researcher's perspective, especially when you're working on these up-and-coming vulnerabilities, I think that the real question is: are you comfortable sharing this information with who you're trying to get help from in terms of the LLM? And so a lot of people just don't want to deal with it, and they'll host it themselves.”▶ 28:20

That privacy wall has accelerated a migration to open-weight models. Teams are standing up local instances of architectures like Qwen and GLM on internal hardware. Once security teams own the weights, they control the refusal behavior. McIntyre pointed out that researchers are taking this a step further through weight abliteration: “In fact, some people have even been talking about abliterating, so they're editing the weights themselves to make them even kind of more dangerous.”▶ 27:16 By mathematically identifying and modifying the direction vectors responsible for refusal responses, teams remove safety tripwires entirely. They get an assistant that inspects raw binary crashes and writes exploit payloads without triggering corporate policy blocks.

What to Do With This

If your engineers handle vulnerability research, penetration testing, or defensive tooling, cut your dependencies on third-party model APIs this week. Stand up an internal inference box running an uncensored open-weight model like Qwen on your own hardware. Audit every prompt pipeline touching unpublished security bugs to ensure zero-day proof-of-concept code never hits an external cloud provider.