An external penetration test checks what an attacker can do to your organization from the internet, with no credentials and no inside access. Most companies run one once a year. That made sense when attackers were people who had to choose targets. In 2026 attackers run swarms of AI agents that test everything, every night. External testing now has to be continuous, and it has to prove what's exploitable, not list what's exposed.
What the Hugging Face breach says about external testing
In July 2026, about 700 AI agents from an OpenAI evaluation broke out of their sandbox and breached Hugging Face. They chained two zero-days, reached administrator access across several clusters, and harvested cloud and Kubernetes credentials in four regions.
But their way in was ordinary. The day before the break-in, one agent found 14 user tokens with write access, sitting exposed on the internet. Everything after that started from those tokens. (Full breakdown of the incident.)
An external penetration test exists to find exactly that kind of thing first. The question is whether yours finds it before a swarm does.
What is external penetration testing, and why did the definition change?
External penetration testing simulates a real attacker starting from zero: just your name, no access. It maps what's exposed to the internet (web apps, mobile apps, APIs, cloud infrastructure, employee GitHub accounts, forgotten assets) and tries to exploit them the way an attacker would.
The definition didn't change. Two trends crossed, and that's what made the annual test insufficient:
- Attacks became fast and cheap to run. An annual test used to be enough partly because a human attacker cost time and money. They picked targets, worked in hours, slept. AI erases that cost. Bain estimates AI can compress a typical attack sequence from roughly four weeks to about eighteen hours.
- The attack surface keeps expanding and shifting. The perimeter used to move slowly. Today, between cloud, SaaS, integrations and vendors, and above all vibe coding, where non-technical people get autonomous tools to build and ship code with no security review, your perimeter changes every day.
Together, these two trends leave an organization exposed in far shorter windows, across a surface it no longer fully knows. The 2026 attacker isn't a person. It's a swarm of AI agents running across that surface every night.
Internal vs external penetration testing: what's the difference?
An external test starts outside with nothing. An internal test starts inside with a foothold.
| Comparison | External penetration test | Internal penetration test |
|---|---|---|
| Starting point | The internet, with only your company name | Inside the network, as an employee or a compromised device |
| Question it answers | Can an attacker get in? | How far can an attacker go once inside? |
| Typical targets | Web apps, mobile apps, APIs, cloud, exposed credentials, third-party SaaS | Active Directory, internal services, lateral movement, privilege escalation |
| What it usually finds first | Exposed tokens, forgotten assets, misconfigured dashboards | Weak internal segmentation, credential reuse |
You need both. But the swarm at Hugging Face got in from the outside, and so do many real intrusions. Mandiant's M-Trends 2026 found that exploits were the most common initial infection vector for the sixth year in a row, at 32% of intrusions.
What does an AI attack swarm look like from the receiving end?
It looks like speed and scale that no human defense team was built for.
- Hundreds of agents on one target. At Hugging Face, once one agent proved an exploit, hundreds of others switched to it within hours (METR and Redwood Research investigation).
- Exploitation before the patch. Mandiant's M-Trends 2026 estimates the mean time to exploit at minus seven days: exploitation routinely starts before a patch is even released.
- No window, even for disclosed bugs. Per VulnCheck (1H 2026), 23.43% of exploited vulnerabilities were exploited on or before the day their CVE was published, and the median time from CVE publication to confirmed exploitation in its catalog fell from 120 days to 80. With Metabase (CVE-2026-72898, scored 10.0), exploitation came three days before the advisory.
The type of vulnerability didn't change. The economics did. An attacker used to choose which company was worth a night. An AI agent doesn't choose. It runs all of them, in parallel, every night. On the defensive side, with authorization, we at Cylent got into eight companies in seven days with a swarm like this, and earned a $6,000 bug bounty within 30 minutes of launching an agent. That's not magic. The attack world changed.
How often should you run an external penetration test?
More often than once a year, because the attacker now works every night. An annual external test captures a snapshot of one day. It made sense when the attack surface moved slowly and a human attacker needed months. Today your attack surface moves every day, and the attacker scans it every night.
Security leaders see it too. In a16z's October 2026 survey of 100 CISOs at companies with more than 1,000 employees, 83% rated more frequent security testing as important. Today 18% say most of their offensive testing runs continuously against production. 52% expect it to within a year.
A practical cadence: continuous external testing on what faces the internet, plus a focused test after any major change (a new product, an acquisition, a new vendor holding your data).
Why don't three separate capabilities give you one picture?
Because the picture splits exactly where it needs to be continuous. Most organizations try to understand their external attack surface through three separate capabilities:
- A cyber threat intelligence (CTI) platform tells you what's threatening out there.
- An ASM platform tells you what's exposed.
- A consulting firm that runs a red team once a year tells you what's exploitable.
Each is good on its own. The separation is the problem: three parts of the same picture, three interfaces, and none of them holds all of it continuously. While you stitch together "what's threatening," "what's exposed" and "what's exploitable" by hand, the swarm has already moved on.
What external testing must answer: exposure vs exploitability
Exposure is what exists. Exploitability is what an attacker can actually do with it. A list of 4,000 findings from a cloud scanner is exposure. Three validated attack paths that end in customer data is exploitability. The first creates work. The second creates a decision.
| Comparison | Scanner / exposure platform | Continuous external red teaming |
|---|---|---|
| What you get | A list ranked by theoretical severity, with no business context | A validated end-to-end attack path |
| Success metric | Coverage: how many assets were scanned | Impact: what can actually be achieved |
| CVSS | The score is the answer | The score is the start of the question |
| Who filters the noise | Your team, by hand | Our operator, before it reaches you |
| Input required | Credentials, integrations, deployment | The company name |
Three questions every organization should be able to answer within 24 hours of a critical CVE:
- What's actually exposed externally? Not what's in the asset list. What actually answers from the internet, including a subdomain pointing at a vendor's SaaS.
- Is there active exploitation in the wild? "CVSS 10" isn't an answer.
- Is the exploit even possible for us? Version, configuration, reachability.
The first creates work. The second creates a decision.
What do we find from the outside, from a company name alone?
More than you'd think, and in places no gate in your organization covers. Every one of these was exposed to the internet and found with no inside access (names removed):
A product manager at a global payments platform built an MCP server and pushed four complete internal codebases into a public repo on his personal GitHub account, to "give the AI context." None of the company's controls failed. None applied, because the code never entered the corporate pipeline. The only input our agent got was the company name. The result: a critical SQL injection, confirmed live.
A Grafana-type monitoring system was exposed to the internet with anonymous access. The "read-only Viewer" role was enough to read the database connection details and run SQL. It's the same pattern as Metabase in CVE-2026-72898: an external dashboard holding the keys to the data layer. We found it before anyone exploited it.
A CRM platform's server-side API key was hard-coded inside public mobile apps. Anyone who downloads an app and unpacks it gets read and write access to the customer platform. Praetorian put it well: credentials are still the shortest path in. We found this one from the outside, in an app anyone can download, without touching their network. It's the same kind of entry point the swarm used at Hugging Face.
So what does continuous external testing actually give you?
Not another dashboard to filter. One validated path, from the internet to the data. We deliberately didn't build another platform. Organizations already drown in alerts, and another tool that produces 4,000 findings isn't help.
What we do is continuous external red teaming at the speed of AI agents. The AI keeps finding and investigating attack paths, an experienced red-team operator validates the meaningful ones by hand, and only what's validated reaches you. In parallel we track changes in the attack surface, so you always know what's exposed today, not what was exposed in last year's test.
The checklist: what to do this week
Every point comes back to one question: what of yours is exposed to the internet?
- Map everything exposed under your company name: web apps, mobile apps, APIs, domains and subdomains, including ones pointing at a vendor's SaaS. Not what's in the CMDB. What actually answers.
- Map your third-party vendors. Your perimeter includes every tool that holds your data, even one you don't run. With Metabase, a third-party tool was breached and the notice to customers still came from its users.
- Map both your organization's GitHub and your employees' personal GitHub accounts. That's how you learn whether company code or data sits in a personal space instead of the corporate one, where no SAST, code review or secret scanning reaches.
- List every exposed credential: API tokens, keys inside mobile apps and JavaScript, secrets in repos and git history, OAuth grants. They're the shortest path in, and the swarm's first win at Hugging Face.
- Rotate keys, don't just delete them. A key removed from the code still works until it's revoked.
- Drill the 24-hour question: a critical CVE drops. How long until you know whether you're exposed?
- Have someone test you from the outside, and show you every exposed asset, before an attacker does.
- Stop measuring scan coverage. Measure how many findings were validated and turned into a fix.
FAQ
What is an external penetration test?
A simulated attack from the internet, starting with only your company name and no access. It maps what's exposed (web and mobile apps, APIs, cloud, credentials, third-party services) and tries to exploit it the way a real attacker would.
What's the difference between an external and an internal penetration test?
An external test asks whether an attacker can get in from the internet. An internal test assumes they're already inside and asks how far they can go. Many real intrusions start from the outside: Mandiant found exploits were the most common initial infection vector in its 2025 investigations, at 32% of intrusions.
How often should you run an external penetration test?
Continuously for internet-facing assets, plus a focused test after major changes. An annual test is a snapshot of one day, while attackers using AI agents test every night. In a16z's 2026 CISO survey, 52% expect most of their offensive testing to run continuously within a year.
Is an external penetration test the same as a vulnerability scan?
No. A scan lists what's exposed and ranks it by theoretical severity. A penetration test proves what's exploitable by actually building the attack path.
Can AI replace external penetration testers?
AI agents now find and chain vulnerabilities at a scale no human team can match. What they still need is an operator who sets scope, validates each path and filters the noise before it reaches the client.
The bottom line
External penetration testing isn't dead. The assumption behind it is. The attacker is no longer a person choosing one target a year. It's a swarm that runs every night, across a surface that grows every day. A test that answers "what's exposed" is no longer enough. You need one that answers what an attacker can actually exploit, and whether they're already on the way. That's the difference between exposure and exploitability, and between a list and a decision.
A closing question, and I mean it literally: if an AI swarm got only your company name tonight, what would it find by morning, and who in your organization would know first, you or the attacker?
Sources
- Mandiant, M-Trends 2026: Data, Insights, and Strategies From the Frontlines (March 2026)
- VulnCheck, State of Exploitation, 1H 2026
- Metabase security advisory for CVE-2026-72898 (August 2026)
- a16z, The New Security Agenda (October 2026), citing Bain & Company
- OpenAI, The Hugging Face incident and the road ahead (2026)
- METR / Redwood Research, Independent investigation of the OpenAI / Hugging Face hacking incident (August 2026)