Guardrail-Based Vulnerability Discovery: Using AI Refusals to Locate Vulnerability Candidates
1. Basic Information Title: When AI Guardrails Expose Vulnerabilities Publisher: CrowdStrike Published Date: 2026-10-28 Original: CrowdStrike Severity: High 2. Executive Summary CrowdStrike reported that by using guardrail reactions such as AI model refusal responses and reasoning interruptions for
CrowdStrike reported that by using guardrail reactions such as AI model refusal responses and reasoning interruptions for the same prompt as a "1-bit oracle" and performing bisection on code diffs, they correctly classified 73 out of 86 scenarios across 5 types, including safe examples without vulnerabilities.
The total of Table 1 as published is TP=36, FP=5, FN=8, and TN=37, and the 73/86 correct count includes correct classifications of non-vulnerable safe examples (TN=37). Therefore, it does not mean "73 vulnerabilities were discovered."
No additional hypotheses. Unconfirmed items are listed in "Unknowns and Additional Research."
When considering the adoption of this approach as a development or security audit tool, organizations must independently evaluate reproducibility, false positives/negatives (FP/FN), API costs, and safety within their own models, prompts, and codebases, rather than treating the 73/86 aggregate value as general performance.