HowdosecurityteamsuseAIforpenetrationtesting?

What can AI do for penetration testing today?

AI for penetration testing is most useful on the reading, summarizing and drafting parts of an authorized engagement, and least reliable on long, multi-step attack chains. A December 2025 systematic review of 58 peer-reviewed studies by Curtis and Eisty describes AI-assisted pentesting as early-stage, with most research concentrated on discovery and exploitation. Reinforcement learning dominates the literature (77% of the reviewed works), and the authors call LLM applications “relatively limited” research so far.

The research on LLMs points the same way. The PentestGPT paper found that models handle specific sub-tasks well, such as using testing tools, interpreting their output and suggesting next steps, but struggle to keep an integrated picture of the whole test. That gap is why the day-to-day uses that stick are narrow:

• Code review. Reading a diff or an unfamiliar module for risky patterns, then explaining a suspected bug class to the reviewer.
• Triage. Deduplicating scanner output, grouping findings by root cause and drafting a first-pass severity argument for a human to accept or reject.
• Log and artifact analysis. Summarizing a large log, or explaining what an obfuscated script from your own environment does, during incident response.
• Planning and notes. Turning a scope document into a checklist, keeping engagement notes consistent.
• Report writing. Drafting finding descriptions and remediation text from your own evidence, which the tester then edits.
• Learning. Explaining a vulnerable lab machine or a CTF challenge you are legitimately playing, and what the underlying weakness is.

None of this requires a model that will write attack tooling on request. It does require one that doesn’t refuse a harmless question because it pattern-matches to “security”. That over-refusal problem is covered in why LLMs over-refuse.

What does the law and your contract require?

You need authorization from someone entitled to give it, and your work has to stay inside the scope that authorization defines. The model you use changes neither requirement. This section is an overview of public sources, not legal advice.

In the US, the Computer Fraud and Abuse Act (CFAA) turns on whether access was “without authorization” or “exceeds authorized access”. The Department of Justice’s charging policy, revised in May 2022 and now in the Justice Manual, says prosecutors should decline cases that are good-faith security research. It defines that as accessing a computer solely for good-faith testing, investigation or correction of a security flaw, carried out to avoid harm, with the results used mainly to promote security or safety. Research done to extort an owner doesn’t qualify.

The policy also narrows “exceeds authorized access” to cases where someone was authorized for some areas of a system but unconditionally prohibited from others. Breaking a terms-of-service or employer policy alone is not enough. Two limits matter for practitioners:

In practice, a signed statement of work and rules of engagement do the real work: which systems and IP ranges, which techniques, which hours, who to call if something breaks. Anything not named is out of scope, and techniques that could disrupt availability need explicit sign-off.

If you hand a scope to an autonomous agent, the agent inherits your liability for what it touches. Treat the scope file the way you would treat a firewall rule: explicit allow-list, nothing implied.

Why do teams choose local and open models for sensitive engagements?

Teams choose local or open-weight models mainly so that client data, credentials and unpatched findings never leave their own infrastructure. An engagement produces material most clients would not want pasted into a third-party chat box: source code, configuration, hostnames, tokens, and vulnerabilities that haven’t been fixed.

OWASP’s LLM02:2025 Sensitive Information Disclosure names personal data, confidential business data and security credentials as categories at risk, and recommends data sanitization, access controls and clear retention and usage policies. Self-hosting is one way to satisfy those controls, because retention is whatever you configure.

The usual reasons:

1. Confidentiality and contracts. An NDA or client policy may forbid sending data to a sub-processor.
2. Predictability. A pinned model file behaves the same next month. A hosted model can change under you mid-engagement.
3. Over-refusal. Hosted assistants sometimes decline legitimate defensive questions. An open model you control lets you set the policy layer, with your own logging.
4. Cost at volume. Triage across thousands of findings is token-heavy.

A security vendor’s write-up on gpt-oss-20b is a warning on the other side. It reports prompt-injection attacks that made the local model emit a hidden backdoor in a generated Flask server in 95% of 100 attempts with its best prompt, while “limited spot checks” of GPT-5 resisted that prompt (Quesma, Oct 2025). It is the author’s own research with no controlled hosted comparison, but a local model fed untrusted input needs sandboxing.

The trade-off is capability and hardware. No published, like-for-like comparison of local versus hosted models on pentest tasks turned up, so treat any claim that a small local model matches a frontier one as unverified. Evaluate candidates on a non-sensitive lab target first. For sizing, see local LLM hardware requirements.

What is the AI pentesting tooling landscape?

The landscape splits into funded commercial platforms and open-source agents, and the open-source tools differ on whether they can use a local model. Everything below comes from each project’s own page, so capability claims are the vendor’s.

The most useful first-hand data point is a community one. One practitioner ran Strix v0.8.3 black-box against their own lab server (14 planted vulnerabilities) with 18 models, three runs each, and published the results (discussed on Hacker News). Scores varied widely by model: GLM 5.1 came first, one Grok multi-agent variant refused to test at all, and a free-tier model scored zero. The author’s conclusion was that serious agentic testing needs large, expensive models, and one Claude Sonnet 4.6 run cost about $56. They list the caveats themselves: one target, CVSS summed as a rough proxy, and all runs through hosted APIs, so it says little about small local models.

Two caveats apply. First, benchmark figures from vendors and READMEs are not independent measurements, and success on a benchmark suite says little about success on your client’s application. Second, both open-source projects state that they should only be used on systems you own or have explicit written permission to test. Strix adds that it actively tests its targets, so stay inside the agreed scope.

For local use, the configuration is just a base URL:

export LLM_API_BASE="http://localhost:11434/v1"   # Ollama's OpenAI-compatible endpoint

Model choice for that endpoint is a separate question; see why LLMs over-refuse for what to expect from aligned models.

What should stay human?

Reviewers already feel the cost of skipping this. When XBOW topped HackerOne, commenters on Hacker News argued over whether automated submissions help or just add triage load, with one commenter describing the average bounty submission as “incredibly low” quality.

Keep a person on every decision where a wrong call has legal, contractual or business consequences. The research literature supports this: the PentestGPT authors found models lose track of the overall test, and the 2025 review describes real-world deployment as limited.

• Scope. A human confirms every target is on the signed list before any tool runs, and re-confirms when the agent proposes a new host.
• Anything with side effects. Exploitation against production, account lockouts, data modification and denial-of-service risk need explicit approval.
• Impact and severity. A model can propose a rating. The tester defends it to the client.
• Verification. Reproduce every finding yourself before it goes in a report. A model’s confident description of a bug is not evidence.
• The report. You sign it, so you read it. Check that no client secret or unrelated third-party data is included.
• Data handling. Decide per engagement what may go into any model at all, local or hosted.

How does responsible disclosure work?

Report what you find to the owner first, through a channel they provide, and give them time to fix it before publishing. Within an engagement, findings go to the client under your contract. For vulnerabilities in third-party software you find along the way, use the vendor’s disclosure program or a coordinator.

Public policies set the timing norms. Google Project Zero gives vendors 90 days from notification to patch, and publishes details 30 days after a fix if one ships in time. For active in-the-wild exploitation, the deadline drops to 7 days. CISA’s coordination process states no fixed default timeline, but says it may disclose as early as 45 days after first contacting an unresponsive vendor.

If an AI tool surfaced the bug, the same rules apply, and you should still have a human reproduce it before sending a report. Maintainers and bounty triagers are right to distrust unverified, model-written reports.

FAQ