Provides a JS library @ds-ukassel/cdns and Python library cdns to identify CDNs.
All CDNs are listed in cdns.yaml in this repository, and dist/cdns.json in the package.
- name: AwesomeCDN
homepage: https://www.awesomecdn.com
type: commercial
domains: # See "Domain patterns" below
- 'awesomecdn.com' # 'awesomecdn.com' is a full match of the domain (e.g. 'awesomecdn.com')
- '**awesomecdn**' # 'awesomecdn' is a substring of the domain (e.g. 'a.awessomecdn.com')
- '*.cdn.awesome.com' # '*' is a wildcard for a single domain level (e.g. 'a.cdn.awesome.com' or 'b.cdn.awesome.com')
- '**.awesomecdn.*' # '**' is a wildcard for one or more domain levels (e.g. 'a.b.awesomecdn.de' or 'c.d.e.awesomecdn.com')
ranges:
- range_type: 'ipv4'
url: 'https://api.awesomecdn.com/ips-v4/'
- range_type: 'ipv6'
url: 'https://api.awesomecdn.com/ips-v6/'
- range_type: 'combined'
url: 'https://api.awesomecdn.com/ips/'
- range_type: 'ipv4'
ranges:
- '111.111.111.111/32'
last_updated: 1970-01-01
asns:
- 12345
- 67890
headers: # See "Header Patterns" below
- X-AwesomeCDN: # 'X-AwesomeCDN' header must be present
- Server: AwesomeCDN # 'Server' header must contain 'AwesomeCDN'
- X-Cache: HIT # 'X-Cache' header must contain 'HIT'
X-Whatever: # AND 'X-Whatever' header must be present
certificates: # Keywords to look for in the certificate other than the CDN name
- "Awesome"
- "Awesome Inc."
When implementing the wildcard syntax for domains, the following rules apply:
*matches a single domain level*.example.commatchesabc.example.com*.example.comdoes not matchabc.def.example.com
**matches one or more domain levels or any prefix**.example.commatchesa.example.comandb.c.example.com**example.commatchesexample.com,a.example.com, andb.c.example.com
* and ** can appear anywhere in the domain string, including the beginning, middle, or end.
-
*is equivalent to[^.]+(matches any character except dots, one or more times) -
**is equivalent to.*(matches any character, including dots, zero or more times) -
All special characters are escaped, e.g.
.turns into\.. -
The regex receives the prefix
^and suffix$.
**example.com->^.*example\.com$*.example.com->^[^.]+\.example\.com$
Header patterns have a few rules:
Header:(JSON:{"Header": null}) means the header must be present, but can have any value.Header: foo(JSON:{"Header": "foo"}) means the header must be present and equal the value.- Multiple patterns in the same entry, e.g.
or JSON:
Header-1: Header-2: bar{"Header-1": null, "Header-2": "bar"}Means ALL header patterns must match. - Header keys are case-insensitive, but values are case-sensitive.
The values can use * as a wildcard for any number of characters, e.g. foo* matches foo, foobar, and foo123.
Any amount of * can be used at different positions.
To use a literal *, escape it with a backslash: \*.
Each CDN can have a designated type field.
The type can either be commercial, private, public_assets or other.
commercial: CDNs used by different customers (e.g. Akamai, Fastly, ...)private: CDNs used internally by a single company (e.g. Meta, Instagram, ...)public_assets: CDNs providing public assets not uploaded by customers (e.g. BootstrapCDN)other: CDNs with an entirely different model
The field is optional as this information isn't always available/clear.
The certificates contain a list of keywords to look for in the certificate of a website.
The name of the CDN should always be searched as well and doesn't have to be included.
def compile_domains(domains: list[str]) -> Pattern:
...
def compile_domain(domain: str) -> Pattern:
...
(source): Compile domain patterns to a RegExp according to the Domain Pattern rules.
def matches(headers: Dict[str, str], matcher: Dict[str, Optional[str]]) -> bool:
...
def match_headers(headers: Dict[str, str], matchers: List[Dict[str, Optional[str]]]) -> Dict[str, str]:
...
(source): Match response headers against the patterns according to the Header Pattern rules.
function compileDomains(domains: string[]): RegExp
function compileDomain(domain: string): RegExp
(source): Compile domain patterns to a RegExp according to the Domain Pattern rules.
function matchHeaders(headers: Record<string, string>, patterns: Record<string, string | null>[]
(source): Match response headers against the patterns according to the Header Pattern rules.
- TurboBytes CDN Finder (MIT)
- CDNProbe (Apache 2.0)
- cname_cache: Parts of the substring list were used to extend our own list
- cdnlist: Some CDNs from the cdn list were used to extend our own list
- CDN Handbook
- Information about some CDNs
- IPInfo
- Information about ASNs used by CDNs
- CDN-CNAME-Domain
- cdn-ip-database (MIT)
- Third Party Web (MIT)
- ProjectDiscovery cdncheck (MIT)
- Netify
- Behnam Shobiri, Mohammad Mannan, and Amr Youssef. 2023. CDNs’ Dark Side: Security Problems in CDN-to-Origin Connections. Digital Threats 4, 1, Article 3 (March 2023), 22 pages. https://doi.org/10.1145/3499428
Original content created by members of the Distributed Systems Group is licensed under the MIT license. The information compiled in this repository has in part be collected from publicly available repositories and is intended for research purposes. Content created by third parties remains subject to its respective license. Please respect the original licensing terms and provide appropriate attribution when using or redistributing third party content. See our references for more information.