AI helped find most of curl's 24 new CVEs. What made them real was the human check.

In January, curl shut down its bug bounty. Daniel Stenberg said the goal was to remove the incentive for people to send "crap and non-well researched reports to us. AI generated or not." The last week of the program brought seven reports and none of them described a vulnerability.

Today, curl 8.23.0 shipped with 24 security fixes. AISLE says its researchers found 18 of them. The only one rated High, a use-after-free in the HTTP/2 upload path, was reported by an Anthropic researcher using Anthropic's new open source scanner. The curl team confirmed it within hours and cut the release cycle from eight weeks to six to ship the fix.

Same project, same kind of tools, nine months apart. One produced so much noise the bounty closed. The other produced real CVEs. I don't think the difference was the model.

Look at how the October reports got to the curl team.

AISLE says a security researcher reviews the complete report before submission. Then curl's own security team decides whether the behavior is real, whether it breaks curl's security policy, how severe it is and whether it gets a CVE. Stenberg describes AISLE's work as "detailed and accurate findings backed up by a team of skilled humans."

So there were two human checks before anything became a CVE. One on the side that ran the tools, one on the side that owns the code.

The January reports skipped the first check. Someone pointed a tool at curl, got something that sounded like a vulnerability and sent it. The cost of verifying moved from the reporter to a small maintainer team. Stenberg's rule was simple: never report a bug unless you actually understand it and can reproduce it.

I audit smart contracts, and AI is now a big part of how my team works. Last week I wrote that without prior experience auditing manually, I don't think AI auditing can be done properly. The curl story is the same point at a much bigger scale.

In my last sweep on a competition codebase I ran six agents in parallel. Five came back with reasoned negatives. The sixth produced passing PoCs, but they only rediscovered a finding we already had. None of that output could be taken at face value.

The two failure modes are the same ones curl lived through:

You need someone who knows the code well enough to catch both. That's the part that didn't get cheaper.

On October 8 Anthropic opened an opt-in scanning service for open source maintainers. They say plainly that its output "will be fully model-generated, without human review or triage." In their own six months of scanning they got over 29,000 candidate vulnerabilities and manually reviewed about 6,000.

I don't think that's bad. It's free, it's opt-in, and maintainers know what they're getting. But it moves the triage to you. If you enroll, the first check from the curl story is your job now.

Whether you maintain a library or you're about to launch a protocol, a few things I'd keep in mind:

curl didn't stop taking AI-assisted reports. It stopped rewarding unverified ones. Now Stenberg says the reviewed work is helping them fix vulnerabilities "at a higher rate than ever before." I think that's the right model for the rest of us.

How are you handling AI scanner output on your own projects? Do you read everything, or do you have a way to rank it first?

Sources: curl 8.23.0 release, the HTTP/2 use-after-free write-up, AISLE's write-up, Anthropic's opt-in scanner, The Register on the bounty shutdown.

I wrote this with help from AI for drafting and editing. The opinions and the audit experience are mine.