AI pentesting is penetration testing in which AI agents plan and run the reconnaissance, probing and attack-path building that human testers used to do by hand. In 2026 agents are strong at breadth. An operator is still needed to set scope, approve sensitive steps, prove impact without causing damage, and judge flaws a machine cannot verify alone.
- Almost every offensive security vendor now ships an agent. Having one is table stakes.
- The real differences are validation and noise: what the agent claimed, what proved it, and who decided how far to go.
- Even XBOW, the first autonomous system to top HackerOne's US leaderboard, had its security team review every report before submission.
- "AI pentesting" (testing with AI) and "pentesting AI" (testing models and agents) are different jobs.
What is AI pentesting?
AI pentesting is a penetration test in which AI agents, not just scripts, decide what to try next. A scanner runs a fixed list of checks against a list of assets and stops. An agent reads what comes back, forms a hypothesis, tests it, and chains small findings into a path an attacker could walk.
You'll also see it called AI penetration testing, automated penetration testing, or an autonomous penetration testing platform. The terms overlap. What matters is how much of the loop the machine runs, and where a person sits in it.
The same search term hides a second meaning:
- AI pentesting uses AI to test your systems: web apps, APIs, cloud, CI pipelines, your external attack surface. That's this guide.
- Pentesting AI tests the models, copilots, MCP servers and agents themselves, against risks like prompt injection and sensitive information disclosure (the top two on the OWASP Top 10 for LLM Applications 2025). Terra lists AI red teaming for "Copilots, LLMs, MCPs, and agent behavior," and Praetorian added LLM endpoints to Hannibal. Different scope, different skills. We cover it in our guide to AI agent security.
Who builds AI pentesting tools in 2026?
Almost everyone in offensive security, which is why "we have an agent" no longer separates anyone. Here's how the main players describe themselves, in their own words, checked on their own sites in November 2026:
| Vendor | What it is, in its own words | Where the human sits, per its own material |
|---|---|---|
| XBOW | "a fully autonomous AI-driven penetration tester"; first autonomous system to reach #1 on HackerOne's US leaderboard (June 2025) | Validator agents confirm exploitability; its security team reviewed HackerOne reports before submission |
| Terra Security | "One Agentic Offensive Security Platform. Every Attack Vector. Continuously Validated." | "Human-on-the-Loop at critical decision points" |
| Praetorian (Hannibal) | The autonomous pentesting capability in the Guard Platform, for external, cloud, web application and LLM attack surfaces | Customers set scope, guardrails, budgets and scan windows |
| Horizon3.ai (NodeZero) | "Safely and autonomously hack your production environment, fix what matters most, verify instantly, and repeat continuously." | Customer-run Hack → Fix → Verify cycle |
| Pentera | "AI-driven exposure validation" that "includes autonomous penetration testing" | Customer-run, "safe-by-design" emulation in production |
| CWG (Zeus), Israel | "Autonomous AI that hunts exploitable risk" | "Expert human oversight on every finding" |
Quoted from each vendor's own site, November 2026. Links in the sources below.
Attackers have the same tools. Anthropic's September 2026 threat report found that public offensive frameworks like PentAGI give agent scaffolding "to anyone who downloads them." And demand is moving: in a16z's October 2026 survey of 100 CISOs at companies with 1,000+ employees, 83% said more frequent testing matters, 18% run continuous testing today, and 52% expect to within a year.
Read the right-hand column again. Nearly every vendor puts a human somewhere. That's the right instinct, and it moves the buying question from "is there a human?" to where exactly does the human sit, and what do they sign off on?
Is AI pentesting just expensive vulnerability scanning?
Sometimes, yes. The critics deserve a fair hearing before I disagree. Edgescan's "Why AI Penetration Testing is Just Expensive Vulnerability Scanning" (November 2025) is blunt: automated tools "cannot perform true penetration testing. They're sophisticated vulnerability scanners with better marketing departments." Business logic flaws, it argues, are invisible to automation.
Where they're right: plenty of products sold as AI pentesting are a scanner with a language model writing the report. And access control is genuinely hard. XBOW's own team says so about IDORs: "Since you can't deterministically validate IDORs, the validator had to reason about them."
Where I disagree: a scanner runs known checks against assets you already listed. In the case below, our agent started from a company name, decided that an employee's personal GitHub account belonged to the attack surface, and read four codebases to find one dangerous line. No scanner has a signature for that.
And look at Edgescan's own model: "92% automation" for routine detection and validation, "8% human validation" for complex, high-risk scenarios. The critic and the agent vendors land in the same place. Machines for breadth, people for proof. The real argument is who proved the finding, and how.
What do autonomous agents actually find?
Breadth: everything that's cheap to check, at a scale no team can match. In our engagements, agents are consistently strong at:
- Mapping an external attack surface from a name: subdomains, cloud assets, mobile apps, employees' public code.
- Reading a lot of code or configuration in one pass and pointing at the dangerous lines.
- Trying hundreds of variations of a known class (injection, SSRF, exposed credentials) without getting bored.
- Proposing chains: this key, plus that endpoint, equals write access.
XBOW's public HackerOne run is the best dataset we have on that breadth against real programs. On its way to the top of the US leaderboard it submitted nearly 1,060 vulnerabilities. At the time of its write-up, 130 were resolved, 303 triaged, 208 marked duplicate, 209 informative and 36 not applicable. In XBOW's words: "All findings were fully automated, though our security team reviewed them pre-submission to comply with HackerOne's policy on automated tools."
That's a strong result, and I'm not citing it to knock XBOW. It's the most honest public look at autonomous output after real triage: many real bugs, a real share of noise, and a human reading every report first.
What still needs an operator?
Four things: scope, the sensitive step, proof without damage, and judgment on flaws a machine can't verify by itself.
- Scope. An agent that finds an employee's personal repository has found something outside any asset list. Someone decides whether looking deeper is allowed.
- The sensitive step. Reading production source, using a found credential, touching a live tenant. A person approves first.
- Proof without damage. Prove every link up to the step that would cause harm, then stop.
- Judgment. Is this access-control flaw real? Is this CVE exploitable in this configuration? That's what turns 4,000 findings into a handful of real risks.
Attackers already work this way. In the campaign Anthropic disrupted in November 2025 (GTG-1002), AI did 80–90% of the work and humans stepped in at perhaps four to six decision points. That's where the outcome gets decided, on offense and on defense.
How do an agent and an operator work one finding?
The agent does the breadth, and an operator signs off at every point where the next step could hurt someone. Two cases we've published, with names removed:
- Input: one company name. No credentials, no access.
- Agent: searched GitHub for employees who write code outside work, matched one profile to a senior product leader, and found four complete internal codebases in his personal public repository, pushed there to give an AI assistant context.
- The gate: the agent didn't start reading. It flagged that this was deployable production source, not documentation, and requested operator approval before the deep scan. The operator reviewed the repository, confirmed scope, and approved.
- Finding: in one pass, the agent found user input interpolated into a
SET time_zonestatement: a stored SQL injection that fired for every user in the tenant. - Proof: our research team ran the path against the client's sandbox. 697 ms baseline, 10,822 ms after injection (exactly
SLEEP(10)), 696 ms after cleanup. - What we didn't do: of six findings, three were confirmed live (the SQL injection in sandbox, the source exposure, and a takeover of an unowned GitHub organisation under a former company name). The other three, including a cross-tenant file download and an SSRF, were documented from source for the client. Not exploited against production. No data read.
- Two public GitHub Actions workflows used
pull_request_targetwith a checkout of the pull request's code, behind an "authorize" gate that enforced nothing. - One pull request from a fork, no privileges, under ten seconds on target. CVSS 10.0.
- Five links proven live, up to the publish workflow running (HTTP 204) with the deploy token loaded.
- The sixth link, capturing the token and publishing malicious packages, was deliberately not executed.
In both cases the proof stopped one step before the damage, and a person chose where that step was. An agent alone would have found the same leads. An operator decides what "proven" means for this client, and stops there.
The agent found the door. A person decided how far through it to go.
Scanner, autonomous agent, or agent plus operator?
They solve different problems, and most programs need more than one.
| Vulnerability scanner | Autonomous agent | Agent + operator | |
|---|---|---|---|
| Starting point | The asset list you give it | An asset list, or a name | A name, plus scope agreed with a person |
| How it works | Fixed checks and signatures | Reasons, chains, retries | Same, with human gates at sensitive steps |
| Chains findings | No | Yes | Yes, and picks which chains are worth proving |
| Access control and business logic | Mostly misses | Finds candidates, can't always verify | A person judges and proves |
| Validation | Version or banner match | Automated validators and PoCs | PoC plus operator sign-off, safe for production |
| Noise reaching your team | High | Lower, depends on the validators | Lowest: only proven paths |
| At the destructive step | Not applicable | Depends on guardrails | Proves every link before it, then stops |
| Something out of scope (an employee's repo) | Not seen | Continues or stops, per configuration | Paused for a scope decision |
| Best for | Hygiene and compliance baselines | Continuous breadth | Continuous breadth with proof you can act on |
Scanners still have a job. But a scanner result, an agent result and an operator-validated result are three different kinds of evidence, and your report should say which one you're reading. For the external side, see external penetration testing.
How do you choose an AI pentesting vendor?
Ask questions whose answers show the process, not the product. These come from our vendor questionnaire for CISOs, adapted for AI pentesting:
- "Show me one finding from start to finish." What the agent saw, what it claimed, what proved it. A working proof beats a severity score.
- "Where does the agent stop and wait for a person?" Ask them to name the steps. If the answer is "it doesn't need to," ask what it does when it finds an employee's personal repository.
- "What does it do at the step that could cause damage?" Using a found credential, publishing a package, writing to production. A good answer: prove every link up to it, stop, document.
- "How do you validate access-control and business-logic findings?" Who reasons about them, a validator model or a person, and what evidence do you get?
- "What share of findings never reach me, and who decided?" Ask for duplicate, informative and false-positive rates from a real engagement.
- "Does it test where breaches actually start?" Open self-registration, end-of-life components, dashboards with anonymous access, secrets in chat, wikis and mobile apps, CI pipelines that run outsiders' code, code employees push to personal accounts. If coverage stops at your IP ranges, it stops before most of what we find.
The answer matters less than the reaction. A vendor who answers question 3 in detail, without hesitating, has a process. One who points you to a certificate doesn't. And ask before you sign: after the contract it's a request, before it's a requirement.
What should you do this week?
- Write down what your current testing may do without asking you, and where it must stop. If nobody knows, that's the first finding.
- Count the findings in your last report that came with a working proof versus a version match.
- Send your vendor the six questions above before the next renewal.
- Check whether your scope includes employees' public code. That's where Shadow Code was.
- Search your GitHub workflows for
pull_request_targetwith a checkout of the pull request head. - Decide whether you need AI pentesting, pentesting of your AI, or both, and scope them separately.
- Name the person who receives a proven critical at 2 a.m. Exploitation timelines are now measured in days.
FAQ
Can AI replace penetration testers?
Not in 2026. Agents now do most of the breadth: mapping, probing, reading code, proposing chains. People still set scope, approve sensitive steps, prove impact safely and judge access-control flaws. Even XBOW's record HackerOne run had its security team review every report before submission.
What's the difference between automated penetration testing and AI pentesting?
Automated penetration testing usually means running predefined attack techniques without a person at the keyboard. AI pentesting adds agents that reason about results, choose the next step and chain findings. Vendors use both terms loosely, so ask how much of the loop the machine actually decides.
Is AI pentesting safe to run against production?
It depends on the guardrails and on who approves the sensitive steps. The safe pattern: prove every link up to the step that could cause damage, stop, and document the rest for your team.
What's the difference between AI pentesting and pentesting AI systems?
AI pentesting uses AI to test your applications and infrastructure. Pentesting AI systems tests the models, copilots, MCP servers and agents themselves, against risks such as prompt injection from the OWASP Top 10 for LLM Applications. Different scope, different skills.
How often should you run an AI pentest?
For your external attack surface, continuously, because attackers run agents around the clock. In a16z's October 2026 survey of 100 CISOs, 18% run continuous testing today and 52% expect to within a year.
The bottom line
Every vendor in this guide has an agent, and so do the attackers. The agent is no longer the question. What matters is what happens between "the agent found something" and "this reaches your team": who checked it, how it was proven, and where somebody decided to stop.
So here's my question: in your current testing, who decides what the agent does when it finds something you never asked it to look for?
Related guides: AI agent swarm attacks · External penetration testing · Red teaming in the AI era
Sources
- Edgescan, Why AI Penetration Testing is Just Expensive Vulnerability Scanning (November 4, 2025)
- XBOW, The Road to Top 1: How XBOW Did It (June 24, 2025)
- XBOW, AI Pentesting: Strengths, Gaps, and Validation (April 8, 2026)
- XBOW, How XBOW Finds IDORs with High Accuracy and Low Noise (May 28, 2026)
- Terra Security, terra.security
- Praetorian, Hannibal Puts Autonomous Penetration Testing Under Your Control (September 23, 2026)
- Horizon3.ai, NodeZero
- Pentera, pentera.io
- CWG, Zeus agentic pentesting platform
- OWASP GenAI Security Project, Top 10 for LLM Applications 2025
- Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign (November 2025)
- Anthropic, Countering misuse of AI: September 2026
- a16z, The New Security Agenda (October 2026)
- Cylent Labs, Shadow Code: an AI built the backdoor. Another AI found it.
- Cylent Labs, One pull request to own the supply chain