From 4df88858a2a8ec44e15c11dcb3daf095937b5cfe Mon Sep 17 00:00:00 2001 From: Diede Claessens Date: Mon, 21 Sep 2026 14:16:51 +0200 Subject: [PATCH 1/4] Validate every resolved address and bind it to the request Fixes CVE-2026-77866 and CVE-2026-77972, which affect every release. Only the first IPv4 address of a host was matched against the reserved ranges and the blocklist, and anything else matched nothing. A host rejected in its IPv4 form was accepted written as an IPv6 address, IPv6 entries in a blocklist never matched, and a host that resolved to no IPv4 address was accepted whatever it pointed at. The host is now resolved once and every address it has, in both families, must pass. IPv4-mapped (::ffff:10.0.0.1) and IPv4-compatible (::10.0.0.1) addresses are checked as the IPv4 address they carry, the reserved list covers the IPv6 blocks IANA marks as not globally reachable, and a host without any address is rejected with the new :unresolved_host rather than let through. The default resolver, SafeURL.DNS, looks up A and AAAA records and bounds each query, which was previously unbounded. Validation also returned a verdict and not the address it approved, so the hostname went to an HTTP client that resolved it a second time. A name under an attacker's control could answer with a public address for the check and an internal one for the request, and the same window opens on its own whenever a name legitimately resolves to different addresses across lookups. pin/2 returns the URL with the validated address in place of the host, plus the hostname, address, scheme and port, so a caller connects to what passed while the name still governs the Host header, the server name and the certificate check. The advisory names the HTTP clients this library ships, and both used the old pattern. SafeURL.HTTPoison.get/3 now pins the request and rebuilds hackney's TLS options from the hostname, because supplying :ssl_options replaces hackney's defaults rather than adding to them; a caller who sets :ssl_options or :insecure keeps their own setup, and validation options go under :safeurl. SafeURL.TeslaMiddleware pins any request without TLS and https on Tesla.Adapter.Hackney; with another adapter it validates but cannot pin, since keeping certificate verification pointed at the hostname is spelled differently by each one, and the docs say so. Both clients refuse :follow_redirect, given directly or through the adapter options, because a redirect names a destination that was never validated; hackney was asked to follow one and reached a link-local metadata address. With Tesla, placing the middleware after Tesla.Middleware.FollowRedirects validates every hop instead. The CI matrix moves to a runner image that still exists, and guides/migrating_to_1.1.md covers the URLs 1.0.0 accepted and this release rejects. The version stays on the 1.x line so that a "~> 1.0" dependency picks the fix up. --- .github/workflows/ci.yml | 8 +- README.md | 79 +++++-- guides/migrating_to_1.1.md | 92 ++++++++ lib/safeurl/client.ex | 64 +++++ lib/safeurl/dns.ex | 42 ++++ lib/safeurl/dns_resolver.ex | 34 ++- lib/safeurl/httpoison.ex | 51 +++- lib/safeurl/safeurl.ex | 311 ++++++++++++++++++++----- lib/safeurl/tesla_middleware.ex | 99 +++++++- mix.exs | 10 +- test/safeurl/httpoison_test.exs | 31 +++ test/safeurl/tesla_middleware_test.exs | 122 ++++++++++ test/safeurl_test.exs | 114 +++++++++ 13 files changed, 961 insertions(+), 96 deletions(-) create mode 100644 guides/migrating_to_1.1.md create mode 100644 lib/safeurl/client.ex create mode 100644 lib/safeurl/dns.ex create mode 100644 test/safeurl/httpoison_test.exs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7f28f24..20ff58b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,18 +9,18 @@ on: jobs: build: name: CI - runs-on: ubuntu-20.04 + runs-on: ubuntu-22.04 strategy: matrix: include: - elixir: '1.14' - otp: '23' + otp: '24' - elixir: '1.17' otp: '27' steps: - - uses: actions/checkout@v2 + - uses: actions/checkout@v4 - name: Set up Elixir uses: erlef/setup-beam@v1 with: @@ -28,7 +28,7 @@ jobs: elixir-version: ${{matrix.elixir}} - name: Restore dependencies cache - uses: actions/cache@v2 + uses: actions/cache@v4 with: path: deps key: ${{ runner.os }}-${{ matrix.elixir }}-${{ matrix.otp }}-mix-${{ hashFiles('**/mix.lock') }} diff --git a/README.md b/README.md index 0c17b12..0cd79a5 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,10 @@ To use SafeURL with your favorite HTTP Client, see the [HTTP Clients][readme-htt CIDR ranges to the blocklist, or alternatively allow specific CIDR ranges to which the application is allowed to make requests. -You can use `allowed?/2` or `validate/2` to check if a URL is safe to call. If you have the -[`HTTPoison`][lib-httpoison] application available, you can also call `get/4` which will +You can use `allowed?/2` or `validate/2` to check if a URL is safe to call, and +[`pin/2`][docs-pin] to get a URL that connects to the very address that passed. If you have the +[`HTTPoison`][lib-httpoison] application available, you can also call +[`SafeURL.HTTPoison.get/3`][docs-get] which will validate the host automatically before making a web request, and return an error otherwise. ```elixir @@ -84,7 +86,7 @@ following options: - `:schemes` - List of allowed URL schemes. Defaults to `["http, "https"]`. - `:dns_module` - Any module that implements the `SafeURL.DNSResolver` behaviour. - Defaults to `DNS` from the [`:dns`][lib-dns] package. + Defaults to `SafeURL.DNS`, which looks up A and AAAA records with the [`:dns`][lib-dns] package. - `:detailed_error` - Return specific error if validation fails. If set to `false`, `validate/2` will return `{:error, :restricted}` regardless of the reason. Defaults to `true`. @@ -109,37 +111,52 @@ While SafeURL already provides a convenient [`SafeURL.HTTPoison.get/3`][docs-get before making GET HTTP requests, you can also write your own wrappers, helpers or middleware to work with the HTTP Client of your choice. +Validating a URL and then handing the hostname to an HTTP client resolves it twice, and the +second lookup can return a different address than the one that was checked. Use +[`pin/2`][docs-pin] so your client connects to the address that passed, and keep the hostname +for the `Host` header, the server name and the certificate check. See +[Pinning the validated address](#pinning-the-validated-address). + ### HTTPoison -For [HTTPoison][lib-httpoison], you can create a wrapper module that validates hosts -before making HTTP requests: +[`SafeURL.HTTPoison.get/3`][docs-get] already pins the request, sends the hostname as the +`host` header and rebuilds hackney's TLS options from that hostname, so the certificate is +still verified against the name. For the other verbs, wrap it the same way: ```elixir defmodule CustomClient do def request(method, url, body, headers \\ [], opts \\ []) do {safeurl_opts, opts} = Keyword.pop(opts, :safeurl, []) - with :ok <- SafeURL.validate(url, safeurl_opts) do - HTTPoison.request(method, url, body, headers, opts) + with {:ok, pinned} <- SafeURL.pin(url, safeurl_opts) do + HTTPoison.request(method, pinned.url, body, host_header(headers, pinned), opts) end end - def get(url, headers \\ [], opts \\ []), do: request(:get, url, "", headers, opts) - def post(url, body, headers \\ [], opts \\ []), do: request(:post, url, body, headers, opts) - # ... + defp host_header(headers, pinned) do + [{"host", pinned.hostname} | Enum.reject(headers, fn {name, _} -> name == "host" end)] + end end ``` +Passing your own `:ssl_options` to hackney replaces its TLS defaults rather than adding to +them, so a wrapper that needs to pin an `https` request should build the full set from the +hostname the way `SafeURL.HTTPoison.get/3` does, or leave `https` to it. + And you can use it as: ```elixir -iex> CustomClient.get("http://230.10.10.10/data.json", [], safeurl: [block_reserved: false], recv_timeout: 500) +iex> CustomClient.request(:get, "http://230.10.10.10/data.json", "", [], safeurl: [block_reserved: false], recv_timeout: 500) {:ok, %HTTPoison.Response{...}} ``` ### Tesla -For [Tesla][lib-tesla], `SafeURL` provides a helper middleware out-of-the-box, which you can plug anywhere you're using `Tesla`: +For [Tesla][lib-tesla], `SafeURL` provides a helper middleware out-of-the-box, which you can plug anywhere you're using `Tesla`. +It pins the request where it can do so without weakening TLS, which means any request without TLS, and +`https` with `Tesla.Adapter.Hackney`. With another adapter an `https` request is validated but not pinned, +so call [`pin/2`][docs-pin] yourself and pass the hostname through that adapter's connect options. +Place it after `Tesla.Middleware.FollowRedirects` if you follow redirects, so that every hop is validated: ```elixir defmodule DocumentService do @@ -157,6 +174,40 @@ end
+## Pinning the validated address + +Validating a hostname and then handing that hostname to an HTTP client resolves it twice, +and the second lookup can return a different address than the one that was checked (DNS +rebinding). `pin/2` returns the URL with the host replaced by the validated address, together +with the original hostname for the `Host` header, SNI and certificate verification: + +```elixir +iex> SafeURL.pin("https://includesecurity.com/robots.txt") +{:ok, + %{ + url: "https://192.0.78.24/robots.txt", + hostname: "includesecurity.com", + address: {192, 0, 78, 24}, + scheme: "https", + port: 443 + }} +``` + +Only the first validated address is returned; the others were validated too. + +With [Req][lib-req], for example: + +```elixir +with {:ok, %{url: url, hostname: hostname}} <- SafeURL.pin(url) do + Req.get(url, connect_options: [hostname: hostname]) +end +``` + +Every address the host resolves to, IPv4 and IPv6, has to pass validation, and a host without +any address is rejected with `:unresolved_host`. + +
+ ## Custom DNS Resolver In some cases you might want to use a custom strategy for DNS resolution. You can do so by @@ -218,8 +269,10 @@ SafeURL is officially maintained by the team at [Slab][slab]. It was originally [includesecurity]: https://github.com/IncludeSecurity [readme-http]: #http-clients [docs]: https://hexdocs.pm/safeurl -[docs-get]: https://hexdocs.pm/safeurl/SafeURL.html#get/4 +[docs-get]: https://hexdocs.pm/safeurl/SafeURL.HTTPoison.html#get/3 [docs-dns]: https://hexdocs.pm/safeurl/SafeURL.DNSResolver.html +[docs-pin]: https://hexdocs.pm/safeurl/SafeURL.html#pin/2 +[lib-req]: https://hexdocs.pm/req [lib-dns]: https://github.com/tungd/elixir-dns [lib-tesla]: https://github.com/elixir-tesla/tesla [lib-httpoison]: https://github.com/edgurgel/httpoison diff --git a/guides/migrating_to_1.1.md b/guides/migrating_to_1.1.md new file mode 100644 index 0000000..79c88fb --- /dev/null +++ b/guides/migrating_to_1.1.md @@ -0,0 +1,92 @@ +# Migrating to v1.1 + +`v1.1.0` fixes two advisories against every earlier release, +[CVE-2026-77866](https://cna.erlef.org/cves/CVE-2026-77866.html) and +[CVE-2026-77972](https://cna.erlef.org/cves/CVE-2026-77972.html). It stays on +the `1.x` line so that `{:safeurl, "~> 1.0"}` picks the fix up, but it does +reject some URLs that `v1.0.0` accepted. All of those were the holes. + +## Every resolved address is checked + +`v1.0.0` looked at the first IPv4 address of a host and treated everything else +as matching no range. An address written in IPv6 form passed even when its IPv4 +form was blocked, IPv6 entries in a blocklist never matched, and a host with no +A record was accepted whatever it pointed at. + +Now the host is resolved once and every address it has, in both families, has +to pass. IPv4-mapped (`::ffff:10.0.0.1`) and IPv4-compatible (`::10.0.0.1`) +addresses are checked as the IPv4 address they carry, and the reserved list +covers the IPv6 blocks IANA marks as not globally reachable. + +**What changes for you:** a host whose AAAA record points into a reserved range +is now rejected, where before only its A record was judged. + +## A host that resolves to nothing is rejected + +`v1.0.0` passed `nil` into the range check for such a host, which matched +nothing and so allowed the request. These now fail with the new error +`{:error, :unresolved_host}`, which is part of `t:SafeURL.error/0`. Handle it +wherever you match on specific errors, or set `detailed_error: false` and match +on `{:error, :restricted}`. + +This also catches hosts your resolver cannot answer for, such as names that +only exist in `/etc/hosts` or internationalised names that were not converted +to their ASCII form first. + +## The default resolver queries A and AAAA + +`:dns_module` now defaults to `SafeURL.DNS` rather than `DNS`. A resolver has +to return every address of every family for the guarantee above to hold, so if +you pass your own, make sure it does. `SafeURL.DNS` also bounds each lookup, +five seconds by default: + +```elixir +config :safeurl, SafeURL.DNS, timeout: :timer.seconds(2) +``` + +The `c:SafeURL.DNSResolver.resolve/1` callback's typespec widened from +`{:error, :inet_res.res_error()}` to `{:error, term()}`. Existing +implementations keep working. + +## Requests go to the address that passed + +`validate/2` only ever returned a verdict, so the hostname was handed to an +HTTP client which resolved it a second time. A name that answered with a public +address for the check could answer with an internal one for the request. + +`SafeURL.pin/2` returns the URL with the validated address in place of the +host, plus the hostname for the `Host` header, the server name and the +certificate check: + +```elixir +with {:ok, %{url: url, hostname: hostname}} <- SafeURL.pin("https://example.com/data") do + Req.get(url, connect_options: [hostname: hostname]) +end +``` + +`SafeURL.HTTPoison.get/3` does this for you. `SafeURL.TeslaMiddleware` does it +for requests without TLS and for `https` with `Tesla.Adapter.Hackney`; with any +other adapter it validates but cannot pin, because keeping certificate +verification pointed at the hostname is spelled differently by each one. + +**What changes for you:** nothing in the call, but the destination is reached +at the checked address. If you pass your own `:ssl_options` or `:insecure` to +`SafeURL.HTTPoison.get/3`, securing the connection stays yours. + +## Redirects are not followed for you + +`SafeURL.HTTPoison.get/3` used to pass `:follow_redirect` straight to hackney, +which then requested whatever destination the server named, without validating +it. That option now raises, whether it is given directly or under `:hackney`. +Follow the redirect yourself by validating the location that came back and +requesting it. + +`SafeURL.TeslaMiddleware` raises the same way when the adapter is configured to +follow redirects itself, because it does that after the middleware has run. Put +the middleware after `Tesla.Middleware.FollowRedirects` instead, so that every +hop passes through it. The other order validates the first request only. + +## The reserved list widened + +`192.0.0.0/29` became `192.0.0.0/24`, which is what IANA reserves. If you route +traffic in `192.0.0.8` through `192.0.0.255`, allow it explicitly. diff --git a/lib/safeurl/client.ex b/lib/safeurl/client.ex new file mode 100644 index 0000000..2a70131 --- /dev/null +++ b/lib/safeurl/client.ex @@ -0,0 +1,64 @@ +defmodule SafeURL.Client do + @moduledoc false + + # Shared by the HTTP clients this library ships. They turn a pinned + # destination into what a client needs so that the request reaches the + # validated address while the hostname still governs what the destination + # sees and what its certificate is checked against. + + # hackney derives its TLS options from the host it is given, so once a request + # is pinned to an address it would offer that address as the server name and + # verify the certificate against it. These are hackney's own defaults, rebuilt + # from the hostname instead. + def hackney_ssl_options(hostname) do + hostname + |> to_charlist() + |> :hackney_ssl.check_hostname_opts() + |> Kernel.++(:hackney_ssl.cipher_opts()) + end + + # The caller has said how the connection should be secured, so nothing is + # rebuilt for them. + def tls_configured?(options) do + Keyword.has_key?(options, :ssl_options) or Keyword.has_key?(options, :insecure) + end + + # A redirect names a destination the server chose, which this library never + # got to validate, so a client that would follow one on its own is refused. + def check_no_redirects!(options, where) do + if Keyword.get(options, :follow_redirect, false) do + raise ArgumentError, """ + :follow_redirect cannot be used with #{where}, because the destination of \ + a redirect is chosen by the server and would be requested without being \ + validated. Follow the redirect yourself, by validating the location that \ + came back and requesting it.\ + """ + end + + :ok + end + + # The destination's own authority, which is the hostname plus the port when + # it is not the scheme's default. + def put_host_header(headers, pinned) when is_map(headers) do + headers + |> Map.reject(&host_header?/1) + |> Map.put("host", authority(pinned)) + end + + def put_host_header(headers, pinned) do + [{"host", authority(pinned)} | Enum.reject(headers, &host_header?/1)] + end + + defp authority(pinned) do + if pinned.port == URI.default_port(pinned.scheme) do + pinned.hostname + else + "#{pinned.hostname}:#{pinned.port}" + end + end + + defp host_header?({name, _value}) do + String.downcase(to_string(name)) == "host" + end +end diff --git a/lib/safeurl/dns.ex b/lib/safeurl/dns.ex new file mode 100644 index 0000000..65ef39c --- /dev/null +++ b/lib/safeurl/dns.ex @@ -0,0 +1,42 @@ +defmodule SafeURL.DNS do + @moduledoc since: "1.1.0" + @moduledoc """ + The default `SafeURL.DNSResolver`: looks up both the A and the AAAA + records of a host with the `:dns` package, IPv4 addresses first. + + A host without records in either family resolves to `{:error, :nxdomain}`. + + Both lookups are bounded by `:timeout` (5 seconds by default), which can + be configured with + + config :safeurl, SafeURL.DNS, timeout: :timer.seconds(2) + + """ + + @behaviour SafeURL.DNSResolver + + @default_timeout :timer.seconds(5) + + @impl SafeURL.DNSResolver + def resolve(host) do + case records(host, :a) ++ records(host, :aaaa) do + [] -> {:error, :nxdomain} + ips -> {:ok, ips} + end + end + + # DNS.resolve/4 waits forever by default, which would hang the caller on a + # server that accepts the query and never answers. + defp records(host, type) do + case DNS.resolve(host, type, [], timeout()) do + {:ok, ips} -> ips + {:error, _reason} -> [] + end + end + + defp timeout do + :safeurl + |> Application.get_env(__MODULE__, []) + |> Keyword.get(:timeout, @default_timeout) + end +end diff --git a/lib/safeurl/dns_resolver.ex b/lib/safeurl/dns_resolver.ex index 14648f1..7b714a4 100644 --- a/lib/safeurl/dns_resolver.ex +++ b/lib/safeurl/dns_resolver.ex @@ -5,10 +5,14 @@ defmodule SafeURL.DNSResolver do implementation of `SafeURL.DNSResolver` in the global or local config. - By default, the `DNS` package is used for resolution, - but you can replace it with a wrapper that uses - different configuration or a completely different - implementation altogether. + By default, `SafeURL.DNS` looks up the A and AAAA records + with the `DNS` package, but you can replace it with a + wrapper that uses different configuration or a completely + different implementation altogether. + + A resolver has to return every address the host resolves + to: `SafeURL` validates all of them and rejects the URL if + any one is not allowed. ## Use-cases @@ -25,6 +29,11 @@ defmodule SafeURL.DNSResolver do only one `c:resolve/1` callback that takes a host and returns a list of resolved IPs. + A resolver must return every address of every family the + host has. Returning only the A records would let a host + whose AAAA record points at an internal address through, + which is what `SafeURL.DNS` avoids by querying both. + As an example, suppose you wanted to use [Cloudflare's DNS](https://1.1.1.1/dns/), you can do that by wrapping `DNS` with your own settings in a new @@ -35,7 +44,17 @@ defmodule SafeURL.DNSResolver do @impl true def resolve(domain) do - DNS.resolve(domain, :a, {"1.1.1.1", 53}, :udp) + with {:ok, ipv4} <- query(domain, :a), + {:ok, ipv6} <- query(domain, :aaaa) do + {:ok, ipv4 ++ ipv6} + end + end + + defp query(domain, type) do + case DNS.resolve(domain, type, {"1.1.1.1", 53}, :udp) do + {:ok, ips} -> {:ok, ips} + {:error, _reason} -> {:ok, []} + end end end @@ -67,7 +86,10 @@ defmodule SafeURL.DNSResolver do def resolve(_domain), do: {:ok, [{192, 168, 1, 99}]} end + A resolver that returns an empty list, or an error, makes + the URL fail validation with `{:error, :unresolved_host}`. + """ - @callback resolve(host :: String.t()) :: {:ok, list()} | {:error, :inet_res.res_error()} + @callback resolve(host :: String.t()) :: {:ok, [:inet.ip_address()]} | {:error, term()} end diff --git a/lib/safeurl/httpoison.ex b/lib/safeurl/httpoison.ex index 88fe3dd..82667ae 100644 --- a/lib/safeurl/httpoison.ex +++ b/lib/safeurl/httpoison.ex @@ -6,6 +6,10 @@ if Code.ensure_loaded?(HTTPoison) do supports `get/3`. """ + alias SafeURL.Client + + @where "SafeURL.HTTPoison.get/3" + @doc since: "1.0.0" @doc """ Validate a URL and execute a GET request using `HTTPoison`. @@ -14,8 +18,23 @@ if Code.ensure_loaded?(HTTPoison) do `HTTPoison`, returning the result directly. Otherwise, it will return error. - `headers` and `options` will be passed directly to - `HTTPoison` when the request is executed. + The request is sent to the address that passed validation rather than to + the hostname, so the name cannot resolve to a different address between the + check and the request (see [`Pinning`](SafeURL.html#module-pinning)). The + hostname is still what the destination sees in the `host` header, and what + hackney offers as the server name and verifies the certificate against. + Supplying your own `:ssl_options` or `:insecure` leaves that setup to you. + + Validation options go under `:safeurl`, the same ones + [`SafeURL.pin/2`](SafeURL.html#pin/2) takes: + + SafeURL.HTTPoison.get(url, [], safeurl: [allowlist: ~w[10.0.0.0/8]]) + + Everything else in `headers` and `options` is passed directly to + `HTTPoison` when the request is executed, except for `:follow_redirect`: a redirect names a + destination that was never validated, so following it would step around + the check. Passing it, directly or under `:hackney`, raises + `ArgumentError`. ## Examples @@ -32,8 +51,32 @@ if Code.ensure_loaded?(HTTPoison) do | {:error, SafeURL.error()} | {:error, :restricted} def get(url, headers \\ [], options \\ []) do - with :ok <- SafeURL.validate(url) do - HTTPoison.get(url, headers, options) + {safeurl_options, options} = Keyword.pop(options, :safeurl, []) + check_no_redirects!(options) + + with {:ok, pinned} <- SafeURL.pin(url, safeurl_options) do + HTTPoison.get( + pinned.url, + Client.put_host_header(headers, pinned), + put_tls_options(options, pinned) + ) + end + end + + # hackney honours the option under :hackney as well as at the top level. + defp check_no_redirects!(options) do + Client.check_no_redirects!(options, @where) + Client.check_no_redirects!(Keyword.get(options, :hackney, []), @where) + end + + defp put_tls_options(options, pinned) do + hackney = Keyword.get(options, :hackney, []) + + if pinned.scheme == "https" and not Client.tls_configured?(hackney) do + ssl_options = Client.hackney_ssl_options(pinned.hostname) + Keyword.put(options, :hackney, Keyword.put(hackney, :ssl_options, ssl_options)) + else + options end end end diff --git a/lib/safeurl/safeurl.ex b/lib/safeurl/safeurl.ex index 14f7ff4..14fc755 100644 --- a/lib/safeurl/safeurl.ex +++ b/lib/safeurl/safeurl.ex @@ -8,7 +8,8 @@ defmodule SafeURL do allowed to make requests. You can use `allowed?/2` or `validate/2` to check if a - URL is safe to call. + URL is safe to call, and `pin/2` to get a URL that connects + to the very address that passed validation. ## Examples @@ -25,6 +26,16 @@ defmodule SafeURL do iex> SafeURL.validate("http://230.10.10.10/", block_reserved: false) :ok + iex> SafeURL.pin("https://includesecurity.com/robots.txt") + {:ok, + %{ + url: "https://192.0.78.24/robots.txt", + hostname: "includesecurity.com", + address: {192, 0, 78, 24}, + scheme: "https", + port: 443 + }} + # If HTTPoison is available: iex> SafeURL.HTTPoison.get("https://10.0.0.1/ssrf.txt") @@ -34,6 +45,31 @@ defmodule SafeURL do {:ok, %HTTPoison.Response{...}} + ## Validation + + The host of the URL is resolved once, and every address it + resolves to (IPv4 and IPv6) has to pass the allowlist or the + blocklist. A host that does not resolve to any address is + rejected with `:unresolved_host` rather than let through. + + IPv4-mapped (`::ffff:10.0.0.1`) and IPv4-compatible (`::10.0.0.1`) + addresses are checked as the IPv4 address they carry. + + + ## Pinning + + Validating a hostname and then handing that hostname to an HTTP + client resolves it twice, and the second lookup can return a + different address than the one that was checked (DNS rebinding). + `pin/2` returns the URL with the host replaced by the validated + address, together with the original hostname, so the client + connects to the checked address and still sends the right `Host` + header, SNI and certificate hostname: + + {:ok, %{url: url, hostname: hostname}} = SafeURL.pin("https://example.com/data") + Req.get!(url, connect_options: [hostname: hostname]) + + ## Options `SafeURL` can be configured to customize and override @@ -52,8 +88,8 @@ defmodule SafeURL do `["http, "https"]`. * `:dns_module` - Any module that implements the - `SafeURL.DNSResolver` behaviour. Defaults to `DNS` from - the `:dns` package. + `SafeURL.DNSResolver` behaviour. Defaults to `SafeURL.DNS`, + which looks up A and AAAA records with the `:dns` package. * `:detailed_error` - Return specific error if validation fails. If set to `false`, `validate/2` will return `{:error, :restricted}` regardless of @@ -89,25 +125,63 @@ defmodule SafeURL do """ - @reserved_ranges [ - "0.0.0.0/8", - "10.0.0.0/8", - "100.64.0.0/10", - "127.0.0.0/8", - "169.254.0.0/16", - "172.16.0.0/12", - "192.0.0.0/29", - "192.0.2.0/24", - "192.88.99.0/24", - "192.168.0.0/16", - "198.18.0.0/15", - "198.51.100.0/24", - "203.0.113.0/24", - "224.0.0.0/4", - "240.0.0.0/4" - ] - - @type error() :: :unsafe_scheme | :unsafe_allowlist | :unsafe_blocklist | :unsafe_reserved + import Bitwise + + # Every block IANA marks as not globally reachable, in both families. + # IPv4-mapped and IPv4-compatible addresses are not listed: normalize/1 + # rewrites them to the IPv4 address they carry before they are matched. + @reserved_ranges Enum.map( + ~w[ + 0.0.0.0/8 + 10.0.0.0/8 + 100.64.0.0/10 + 127.0.0.0/8 + 169.254.0.0/16 + 172.16.0.0/12 + 192.0.0.0/24 + 192.0.2.0/24 + 192.88.99.0/24 + 192.168.0.0/16 + 198.18.0.0/15 + 198.51.100.0/24 + 203.0.113.0/24 + 224.0.0.0/4 + 240.0.0.0/4 + ::/128 + ::1/128 + ::ffff:0:0:0/96 + 64:ff9b::/96 + 64:ff9b:1::/48 + 100::/64 + 100:0:0:1::/64 + 2001::/23 + 2001:db8::/32 + 2002::/16 + 2620:4f:8000::/48 + 3fff::/20 + 5f00::/16 + fc00::/7 + fe80::/10 + fec0::/10 + ff00::/8 + ], + &InetCidr.parse_cidr!/1 + ) + + @type error() :: + :unsafe_scheme + | :unsafe_allowlist + | :unsafe_blocklist + | :unsafe_reserved + | :unresolved_host + + @type pinned() :: %{ + url: binary(), + hostname: binary(), + address: :inet.ip_address(), + scheme: binary(), + port: :inet.port_number() + } # Public API # ---------- @@ -116,7 +190,7 @@ defmodule SafeURL do Validate a string URL against a blocklist or allowlist. This method checks if a URL is safe to be called by looking at - its scheme and resolved IP address, and matching it against + its scheme and resolved IP addresses, and matching them against reserved CIDR ranges, and any provided allowlist/blocklist. Returns `true` if the URL meets the requirements, @@ -150,11 +224,13 @@ defmodule SafeURL do Alternative method of validating a URL, returning result tuple instead of booleans. - This calls `allowed?/2` underneath to check if a URL is safe to - be called. If it is, it returns `:ok`, otherwise an error tuple with a + If the URL is safe, it returns `:ok`, otherwise an error tuple with a specific reason. If `:detailed_error` is set to `false`, the error is always `{:error, :restricted}`. + Prefer `pin/2` when you are about to make the request yourself, see + [`Pinning`](#module-pinning). + ## Examples iex> SafeURL.validate("https://includesecurity.com") @@ -173,41 +249,96 @@ defmodule SafeURL do """ @spec validate(binary(), Keyword.t()) :: :ok | {:error, error()} | {:error, :restricted} def validate(url, opts \\ []) do - uri = URI.parse(url) - opts = build_options(opts) - address = resolve_address(uri.host, opts.dns_module) + with {:ok, _pinned} <- pin(url, opts) do + :ok + end + end + + @doc """ + Validate a URL and return it pinned to the address that passed. - result = - cond do - uri.scheme not in opts.schemes -> - {:error, :unsafe_scheme} + The returned `:url` has the host replaced by the first validated + address (the others are validated too, but not returned), `:hostname` + is the host as it was in the URL, for the `Host` header, SNI and + certificate verification, and `:address`, `:scheme` and `:port` are the + parts a client needs to open the connection itself. Connecting to `:url` instead + of the original one makes sure the request reaches the address that + was checked, see [`Pinning`](#module-pinning). - opts.allowlist != [] -> - if ip_in_ranges?(address, opts.allowlist), do: :ok, else: {:error, :unsafe_allowlist} + Errors are the same as for `validate/2`. - opts.blocklist != [] and ip_in_ranges?(address, opts.blocklist) -> - {:error, :unsafe_blocklist} + ## Examples - opts.block_reserved and ip_in_ranges?(address, @reserved_ranges) -> - {:error, :unsafe_reserved} + iex> SafeURL.pin("https://includesecurity.com/robots.txt") + {:ok, + %{ + url: "https://192.0.78.24/robots.txt", + hostname: "includesecurity.com", + address: {192, 0, 78, 24}, + scheme: "https", + port: 443 + }} + + iex> SafeURL.pin("https://[::1]/") + {:error, :unsafe_reserved} - true -> - :ok - end + ## Options - with {:error, _} <- result do - if opts.detailed_error, do: result, else: {:error, :restricted} - end + See [`Options`](#module-options) section above. + + """ + @doc since: "1.1.0" + @spec pin(binary(), Keyword.t()) :: {:ok, pinned()} | {:error, error()} | {:error, :restricted} + def pin(url, opts \\ []) do + options = build_options(opts) + + url + |> URI.parse() + |> validate_and_pin(options) + |> obscure_reason(options) end # Private Helpers # --------------- + defp validate_and_pin(uri, opts) do + with :ok <- validate_scheme(uri.scheme, opts), + {:ok, addresses} <- resolve_addresses(uri.host, opts.dns_module), + :ok <- validate_addresses(addresses, opts) do + {:ok, pinned(uri, hd(addresses))} + end + end + + # Only the first address is handed back, but every one of them was checked, + # so a caller that reaches for another is not reaching past validation. + defp pinned(uri, address) do + host = + address + |> :inet.ntoa() + |> List.to_string() + + %{ + url: URI.to_string(%{uri | host: host}), + hostname: uri.host, + address: address, + scheme: uri.scheme, + port: uri.port + } + end + + # With :detailed_error off, a caller learns that the URL was refused and not + # what about it was refused. + defp obscure_reason(result, opts) do + with {:error, _reason} <- result do + if opts.detailed_error, do: result, else: {:error, :restricted} + end + end + # Return a map of calculated options defp build_options(opts) do schemes = get_option(opts, :schemes) - allowlist = get_option(opts, :allowlist) - blocklist = get_option(opts, :blocklist) + allowlist = parse_ranges(get_option(opts, :allowlist)) + blocklist = parse_ranges(get_option(opts, :blocklist)) dns_module = get_option(opts, :dns_module) block_reserved = get_option(opts, :block_reserved) detailed_error = get_option(opts, :detailed_error) @@ -227,31 +358,83 @@ defmodule SafeURL do defp get_option(opts, key), do: Keyword.get_lazy(opts, key, fn -> Application.get_env(:safeurl, key) end) - # Resolve hostname in DNS to an IP address (if not already an IP) - defp resolve_address(hostname, dns_module) do - hostname - |> to_charlist() - |> :inet.parse_address() - |> case do + # Parsed once per call rather than once per address, like @reserved_ranges. + defp parse_ranges(ranges), do: Enum.map(ranges, &InetCidr.parse_cidr!/1) + + defp validate_scheme(scheme, opts) do + if scheme in opts.schemes, do: :ok, else: {:error, :unsafe_scheme} + end + + # Resolve hostname in DNS to its IP addresses (if not already an IP). + # A host without any address is an error: letting it through would + # let the HTTP client connect to whatever it resolves to later. + defp resolve_addresses(hostname, _dns_module) when hostname in [nil, ""] do + {:error, :unresolved_host} + end + + defp resolve_addresses(hostname, dns_module) do + case :inet.parse_address(to_charlist(hostname)) do {:ok, ip} -> - ip + {:ok, [normalize(ip)]} {:error, :einval} -> - # TODO: safely handle multiple IPs/round-robin DNS - case dns_module.resolve(hostname) do - {:ok, [ip | _]} -> ip - {:error, _reason} -> nil - end + resolve_in_dns(hostname, dns_module) end end - defp ip_in_ranges?({_, _, _, _} = addr, ranges) when is_list(ranges) do - Enum.any?(ranges, fn range -> - range - |> InetCidr.parse_cidr!() - |> InetCidr.contains?(addr) + defp resolve_in_dns(hostname, dns_module) do + case dns_module.resolve(hostname) do + {:ok, [_ | _] = ips} -> {:ok, Enum.map(ips, &normalize/1)} + _no_address -> {:error, :unresolved_host} + end + end + + # An IPv4-mapped (::ffff:a.b.c.d) or IPv4-compatible (::a.b.c.d) address + # reaches the IPv4 host it carries, so it is checked as that IPv4 address. + # The guard leaves out the unspecified and loopback addresses, which share + # the compatible shape without carrying an IPv4 host. + defp normalize({0, 0, 0, 0, 0, 0, high, low}) when high > 0 or low > 1 do + embedded_ipv4(high, low) + end + + defp normalize({0, 0, 0, 0, 0, 0xFFFF, high, low}) do + embedded_ipv4(high, low) + end + + defp normalize(ip), do: ip + + defp embedded_ipv4(high, low) do + {high >>> 8, high &&& 0xFF, low >>> 8, low &&& 0xFF} + end + + # Every address the host resolves to has to pass, otherwise a host with one + # public and one internal address gets through on the public one. + defp validate_addresses(addresses, opts) do + Enum.reduce_while(addresses, :ok, fn address, :ok -> + case validate_address(address, opts) do + :ok -> {:cont, :ok} + error -> {:halt, error} + end end) end - defp ip_in_ranges?(_addr, _ranges), do: false + defp validate_address(address, opts) do + cond do + opts.allowlist != [] -> + if ip_in_ranges?(address, opts.allowlist), do: :ok, else: {:error, :unsafe_allowlist} + + opts.blocklist != [] and ip_in_ranges?(address, opts.blocklist) -> + {:error, :unsafe_blocklist} + + opts.block_reserved and ip_in_ranges?(address, @reserved_ranges) -> + {:error, :unsafe_reserved} + + true -> + :ok + end + end + + defp ip_in_ranges?(address, ranges) do + Enum.any?(ranges, &InetCidr.contains?(&1, address)) + end end diff --git a/lib/safeurl/tesla_middleware.ex b/lib/safeurl/tesla_middleware.ex index 5d5ef1f..88492d7 100644 --- a/lib/safeurl/tesla_middleware.ex +++ b/lib/safeurl/tesla_middleware.ex @@ -8,12 +8,107 @@ if Code.ensure_loaded?(Tesla) do iex> Tesla.client([SafeURL.TeslaMiddleware]) |> Tesla.get("http://localhost/") {:error, :unsafe_reserved} + + ## Pinning + + Where it can, the middleware also sends the request to the address that + passed validation instead of to the hostname, so the name cannot resolve to + a different address between the check and the request (see + [`Pinning`](SafeURL.html#module-pinning)). The destination still sees the + hostname in the `Host` header. + + Doing that over TLS means telling the adapter to keep verifying the + certificate against the hostname, which every adapter spells differently, + so pinning applies to: + + * any request without TLS, whatever the adapter, and + * `Tesla.Adapter.Hackney` over TLS. + + With any other adapter an `https` request is validated but not pinned, and + the rebinding window stays open. To close it there, call `SafeURL.pin/2` + yourself and pass the hostname through your adapter's connect options. + + ## Redirects + + A redirect names a destination the server chose, so it has to be validated + like any other. Place this middleware *after* `Tesla.Middleware.FollowRedirects` + and every hop passes through it: + + plug Tesla.Middleware.FollowRedirects + plug SafeURL.TeslaMiddleware + + The other order validates the first request only. """ @behaviour Tesla.Middleware + alias SafeURL.Client + + @where "SafeURL.TeslaMiddleware" + def call(%Tesla.Env{url: url} = env, next, options) do - with :ok <- SafeURL.validate(url, options) do - Tesla.run(env, next) + check_no_redirects!(env) + + with {:ok, pinned} <- SafeURL.pin(url, options) do + env + |> pin(pinned) + |> Tesla.run(next) + end + end + + # An adapter that follows redirects itself does so after this middleware + # has run, so the destination it picks is never validated. It can be asked + # to on the client or on the single request. + defp check_no_redirects!(env) do + {_module, options} = adapter(env) + + Client.check_no_redirects!(options, @where) + Client.check_no_redirects!(Keyword.get(env.opts, :adapter, []), @where) + end + + defp pin(env, pinned) do + {module, adapter_options} = adapter(env) + + cond do + pinned.scheme != "https" -> repoint(env, pinned, []) + module != Tesla.Adapter.Hackney -> env + Client.tls_configured?(adapter_options) -> repoint(env, pinned, []) + true -> repoint(env, pinned, ssl_options: Client.hackney_ssl_options(pinned.hostname)) + end + end + + # A caller who configured TLS themselves, on the client or on the request, + # keeps it, the same way supplying :ssl_options to + # SafeURL.HTTPoison.get/3 leaves that setup to them. + defp repoint(env, pinned, tls_options) do + %{ + env + | url: pinned.url, + headers: Client.put_host_header(env.headers, pinned), + opts: put_adapter_options(env.opts, tls_options) + } + end + + defp put_adapter_options(opts, []) do + opts + end + + defp put_adapter_options(opts, tls_options) do + Keyword.update(opts, :adapter, tls_options, &Keyword.merge(tls_options, &1)) + end + + # Tesla resolves the adapter before the middleware stack runs, so the + # client carries the one this request will actually use, along with the + # options it was configured with. + defp adapter(env) do + case env.__client__ do + %Tesla.Client{adapter: {module, _fun, [options]}} when is_list(options) -> + {module, options} + + %Tesla.Client{adapter: {module, _fun, _args}} -> + {module, []} + + _other -> + {nil, []} end end end diff --git a/mix.exs b/mix.exs index 3793174..8456aea 100644 --- a/mix.exs +++ b/mix.exs @@ -3,7 +3,7 @@ defmodule SafeURL.MixProject do @app :safeurl @name "SafeURL" - @version "1.0.0" + @version "1.1.0" @github "https://github.com/slab/safeurl-elixir" def project do @@ -24,7 +24,11 @@ defmodule SafeURL.MixProject do source_url: @github, homepage_url: @github, canonical: "https://hexdocs.pm/#{@app}", - extras: ["README.md", "guides/migrating_to_1.0.md"] + extras: [ + "README.md", + "guides/migrating_to_1.1.md", + "guides/migrating_to_1.0.md" + ] ] ] end @@ -44,7 +48,7 @@ defmodule SafeURL.MixProject do block_reserved: true, blocklist: [], allowlist: [], - dns_module: DNS, + dns_module: SafeURL.DNS, detailed_error: true ] end diff --git a/test/safeurl/httpoison_test.exs b/test/safeurl/httpoison_test.exs new file mode 100644 index 0000000..b844b1d --- /dev/null +++ b/test/safeurl/httpoison_test.exs @@ -0,0 +1,31 @@ +defmodule SafeURL.HTTPoisonTest do + use ExUnit.Case, async: true + + test "refuses to validate a request whose redirects it cannot follow" do + assert_raise ArgumentError, ~r/:follow_redirect cannot be used/, fn -> + SafeURL.HTTPoison.get("https://includesecurity.com/", [], follow_redirect: true) + end + end + + test "refuses redirects asked for through the hackney options" do + assert_raise ArgumentError, ~r/:follow_redirect cannot be used/, fn -> + SafeURL.HTTPoison.get("https://includesecurity.com/", [], hackney: [follow_redirect: true]) + end + end + + test "rejects an unsafe URL before any request is made" do + assert {:error, :unsafe_reserved} = SafeURL.HTTPoison.get("https://10.0.0.1/ssrf.txt") + end + + test "validates with the options the caller passes" do + options = [safeurl: [dns_module: TestDNSResolver, allowlist: ["127.0.0.0/16"]]] + + assert {:error, %HTTPoison.Error{}} = + SafeURL.HTTPoison.get("http://allowlisted:9/", [], options) + + assert {:error, :unsafe_reserved} = + SafeURL.HTTPoison.get("http://allowlisted:9/", [], + safeurl: [dns_module: TestDNSResolver] + ) + end +end diff --git a/test/safeurl/tesla_middleware_test.exs b/test/safeurl/tesla_middleware_test.exs index 04d132f..0a47967 100644 --- a/test/safeurl/tesla_middleware_test.exs +++ b/test/safeurl/tesla_middleware_test.exs @@ -5,6 +5,14 @@ defmodule SafeURL.TeslaMiddlewareTest do alias SafeURL.TeslaMiddleware + defmodule RedirectDNSResolver do + @behaviour SafeURL.DNSResolver + + @impl true + def resolve("public.example"), do: {:ok, [{93, 184, 216, 34}]} + def resolve(_host), do: {:ok, [{10, 0, 0, 1}]} + end + test "can pass options" do Tesla.Mock.mock(fn _ -> %Tesla.Env{status: 200} end) @@ -17,6 +25,120 @@ defmodule SafeURL.TeslaMiddlewareTest do assert {:ok, %{status: 200}} = Tesla.get(client, "http://blocked_but_allowlisted") end + test "sends the request to the validated address and keeps the hostname" do + Tesla.Mock.mock(fn env -> + assert env.url == "http://127.0.0.1/data" + assert Tesla.get_header(env, "host") == "blocked_but_allowlisted" + %Tesla.Env{status: 200} + end) + + client = allowlisted_client() + + assert {:ok, %{status: 200}} = Tesla.get(client, "http://blocked_but_allowlisted/data") + end + + test "replaces a host header the caller set" do + Tesla.Mock.mock(fn env -> + assert Tesla.get_headers(env, "host") == ["blocked_but_allowlisted"] + %Tesla.Env{status: 200} + end) + + client = allowlisted_client() + + assert {:ok, %{status: 200}} = + Tesla.get(client, "http://blocked_but_allowlisted/data", + headers: [{"Host", "spoofed"}] + ) + end + + test "leaves an https request alone when the adapter cannot keep verifying the hostname" do + Tesla.Mock.mock(fn env -> + assert env.url == "https://blocked_but_allowlisted/data" + assert Tesla.get_header(env, "host") == nil + %Tesla.Env{status: 200} + end) + + client = allowlisted_client() + + assert {:ok, %{status: 200}} = Tesla.get(client, "https://blocked_but_allowlisted/data") + end + + defp allowlisted_client do + Tesla.client( + [{TeslaMiddleware, [allowlist: ["127.0.0.0/16"], dns_module: TestDNSResolver]}], + Tesla.Mock + ) + end + + test "validates every hop when it runs after the redirect middleware" do + Tesla.Mock.mock(fn + %{url: "http://93.184.216.34/start"} = env -> + assert Tesla.get_header(env, "host") == "public.example" + %Tesla.Env{status: 301, headers: [{"location", "http://internal.example/next"}]} + + %{url: url} -> + flunk("the redirect to #{url} was requested without being validated") + end) + + client = + Tesla.client( + [ + Tesla.Middleware.FollowRedirects, + {TeslaMiddleware, dns_module: RedirectDNSResolver} + ], + Tesla.Mock + ) + + assert {:error, :unsafe_reserved} = Tesla.get(client, "http://public.example/start") + end + + test "refuses an adapter that follows redirects on its own" do + client = + Tesla.client( + [{TeslaMiddleware, dns_module: RedirectDNSResolver}], + {Tesla.Adapter.Hackney, follow_redirect: true} + ) + + assert_raise ArgumentError, ~r/:follow_redirect cannot be used/, fn -> + Tesla.get(client, "http://public.example/start") + end + end + + test "sends the port in the host header when it is not the default" do + Tesla.Mock.mock(fn env -> + assert env.url == "http://93.184.216.34:8080/data" + assert Tesla.get_header(env, "host") == "public.example:8080" + %Tesla.Env{status: 200} + end) + + client = Tesla.client([{TeslaMiddleware, dns_module: RedirectDNSResolver}], Tesla.Mock) + + assert {:ok, %{status: 200}} = Tesla.get(client, "http://public.example:8080/data") + end + + test "rebuilds the TLS options of a hackney client from the hostname" do + env = hackney_env([]) + + assert {:ok, pinned} = TeslaMiddleware.call(env, [], dns_module: RedirectDNSResolver) + assert pinned.url == "https://93.184.216.34/data" + assert pinned.opts[:adapter][:ssl_options] != nil + end + + test "keeps TLS options the client was configured with" do + env = hackney_env(ssl_options: [verify: :verify_none]) + + assert {:ok, pinned} = TeslaMiddleware.call(env, [], dns_module: RedirectDNSResolver) + assert pinned.url == "https://93.184.216.34/data" + assert pinned.opts[:adapter] == nil + end + + defp hackney_env(adapter_options) do + %Tesla.Env{ + url: "https://public.example/data", + __client__: %Tesla.Client{adapter: {Tesla.Adapter.Hackney, :call, [adapter_options]}} + } + end + test "works with other middleware" do client = Tesla.client([Tesla.Middleware.Logger, {TeslaMiddleware, dns_module: TestDNSResolver}]) diff --git a/test/safeurl_test.exs b/test/safeurl_test.exs index 6dacd7d..966e4a0 100644 --- a/test/safeurl_test.exs +++ b/test/safeurl_test.exs @@ -5,6 +5,14 @@ defmodule SafeURLTest do @behaviour SafeURL.DNSResolver @impl true + def resolve("mixed.example"), do: {:ok, [{192, 0, 78, 24}, {10, 0, 0, 1}]} + def resolve("v6-only.example"), do: {:ok, [{0x2606, 0x4700, 0, 0, 0, 0, 0, 0x1111}]} + + def resolve("v6-internal.example"), + do: {:ok, [{192, 0, 78, 24}, {0xFD00, 0, 0, 0, 0, 0, 0, 1}]} + + def resolve("empty.example"), do: {:ok, []} + def resolve("missing.example"), do: {:error, :nxdomain} def resolve(_domain), do: {:ok, [{192, 0, 78, 24}]} end @@ -28,6 +36,83 @@ defmodule SafeURLTest do assert {:error, :unsafe_reserved} = SafeURL.validate("http://192.168.1.1/") end + test "returns false for reserved IPv6 ranges" do + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[fd00::1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[fe80::1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[64:ff9b::a00:1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[2002:a00:1::]/") + end + + test "checks an IPv4-mapped IPv6 address as the IPv4 address it carries" do + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::ffff:10.0.0.1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::ffff:a00:1]/") + + assert {:error, :unsafe_blocklist} = + SafeURL.validate("http://[::ffff:5.5.5.5]/", blocklist: ["5.5.0.0/16"]) + + assert :ok = SafeURL.validate("http://[::ffff:8.8.8.8]/") + end + + test "checks an IPv4-compatible IPv6 address as the IPv4 address it carries" do + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::10.0.0.1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::127.0.0.1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::169.254.169.254]/") + + assert {:error, :unsafe_blocklist} = + SafeURL.validate("http://[::5.5.5.5]/", blocklist: ["5.5.0.0/16"]) + + assert :ok = SafeURL.validate("http://[::8.8.8.8]/") + end + + test "keeps the unspecified and loopback addresses out of the IPv4 mapping" do + assert {:ok, %{address: {0, 0, 0, 0, 0, 0, 0, 0}}} = + SafeURL.pin("http://[::]/", block_reserved: false) + + assert {:ok, %{address: {0, 0, 0, 0, 0, 0, 0, 1}}} = + SafeURL.pin("http://[::1]/", block_reserved: false) + + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::1]/") + end + + test "blocks the IPv4-translated prefix" do + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::ffff:0:a00:1]/") + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[::ffff:0:808:808]/") + end + + test "blocks the IPv6 blocks IANA marks as not globally reachable" do + for address <- ~w[ + 100:0:0:1::1 2001::1 2001:1::1 2001:1::2 2001:1::3 2001:2::1 2001:3::1 + 2001:4:112::1 2001:10::1 2001:20::1 2001:30::1 2001:db8::1 2002:a00:1:: + 2620:4f:8000::1 3fff::1 5f00::1 fc00::1 fe80::1 fec0::1 ff00::1 + ] do + assert {:error, :unsafe_reserved} = SafeURL.validate("http://[#{address}]/") + end + end + + test "allows routed public IPv6 addresses" do + assert :ok = SafeURL.validate("http://[2606:4700:4700::1111]/") + assert :ok = SafeURL.validate("http://[2001:200::1]/") + assert :ok = SafeURL.validate("http://[2a00:1450:4001:800::200e]/") + end + + test "every resolved address has to pass" do + opts = [dns_module: TestDNSResolver] + assert {:error, :unsafe_reserved} = SafeURL.validate("http://mixed.example", opts) + assert {:error, :unsafe_reserved} = SafeURL.validate("http://v6-internal.example", opts) + assert :ok = SafeURL.validate("http://v6-only.example", opts) + end + + test "rejects a host without any address" do + opts = [dns_module: TestDNSResolver] + assert {:error, :unresolved_host} = SafeURL.validate("http://empty.example", opts) + assert {:error, :unresolved_host} = SafeURL.validate("http://missing.example", opts) + assert {:error, :unresolved_host} = SafeURL.validate("http:///path", opts) + assert {:error, :unresolved_host} = SafeURL.validate("http://", opts) + end + test "returns true for reserved ranges if overridden" do opts = [block_reserved: false] @@ -62,4 +147,33 @@ defmodule SafeURLTest do assert {:error, :restricted} = SafeURL.validate("http://0.0.0.0/", opts) end end + + describe "pin/2" do + test "replaces the host with the validated address and keeps the hostname" do + assert {:ok, + %{ + url: "https://192.0.78.24:8443/data?x=1", + hostname: "includesecurity.com", + address: {192, 0, 78, 24} + }} = + SafeURL.pin("https://includesecurity.com:8443/data?x=1", + dns_module: TestDNSResolver + ) + end + + test "keeps an address literal" do + assert {:ok, %{url: "http://3.3.3.3/", hostname: "3.3.3.3", address: {3, 3, 3, 3}}} = + SafeURL.pin("http://3.3.3.3/") + end + + test "brackets an IPv6 address" do + assert {:ok, %{url: "https://[2606:4700::1111]/", hostname: "v6-only.example"}} = + SafeURL.pin("https://v6-only.example/", dns_module: TestDNSResolver) + end + + test "returns the validation error" do + assert {:error, :unsafe_reserved} = SafeURL.pin("http://10.0.0.1/") + assert {:error, :restricted} = SafeURL.pin("http://10.0.0.1/", detailed_error: false) + end + end end From 90987627daf7eb8d94f612eb0dfc3db424700b7f Mon Sep 17 00:00:00 2001 From: Diede Claessens Date: Tue, 22 Sep 2026 10:01:06 +0200 Subject: [PATCH 2/4] Declare the hackney floor the TLS rebuild needs, and ship the guides SafeURL.Client.hackney_ssl_options/1 calls :hackney_ssl.check_hostname_opts/1, which is exported but undocumented and does not exist before hackney 1.16.0. httpoison pulls a newer one in practice, but Tesla's hackney adapter allows ~> 1.6, so an https request pinned through the middleware could raise UndefinedFunctionError at request time instead of being sent. hackney joins the optional dependencies with that floor, so the resolver refuses the combination rather than the request. The package shipped mix.exs, lib and README.md only, while docs: extras points at the migration guides, so they were listed in the docs and missing from the tarball. Both guides ship now. Covers the client helpers too: the host header replaces one the caller set and carries the port when it is not the scheme default, the rebuilt TLS options verify against the hostname rather than the pinned address, and :follow_redirect raises. --- mix.exs | 5 +++- test/safeurl/client_test.exs | 55 ++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+), 1 deletion(-) create mode 100644 test/safeurl/client_test.exs diff --git a/mix.exs b/mix.exs index 8456aea..4b0c7e6 100644 --- a/mix.exs +++ b/mix.exs @@ -57,6 +57,9 @@ defmodule SafeURL.MixProject do defp deps do [ {:httpoison, "~> 1.0 or ~> 2.0", optional: true}, + # SafeURL.Client rebuilds hackney's TLS defaults from the hostname, which + # needs :hackney_ssl.check_hostname_opts/1, exported since 1.16.0. + {:hackney, "~> 1.16", optional: true}, {:inet_cidr, "~> 1.0 and >= 1.0.6"}, {:dns, "~> 2.4"}, {:tesla, "~> 1.0", optional: true}, @@ -74,7 +77,7 @@ defmodule SafeURL.MixProject do name: @app, maintainers: ["Slab"], licenses: ["BSD-3-Clause"], - files: ~w(mix.exs lib README.md), + files: ~w(mix.exs lib guides README.md), links: %{ "Github" => @github, "Slab" => "https://slab.com/" diff --git a/test/safeurl/client_test.exs b/test/safeurl/client_test.exs new file mode 100644 index 0000000..a07805c --- /dev/null +++ b/test/safeurl/client_test.exs @@ -0,0 +1,55 @@ +defmodule SafeURL.ClientTest do + use ExUnit.Case, async: true + + alias SafeURL.Client + + @pinned %{ + url: "https://93.184.216.34/data", + hostname: "public.example", + address: {93, 184, 216, 34}, + scheme: "https", + port: 443 + } + + describe "put_host_header/2" do + test "sets the hostname and drops one the caller sent" do + headers = [{"Host", "spoofed"}, {"accept", "application/json"}] + + assert Client.put_host_header(headers, @pinned) == + [{"host", "public.example"}, {"accept", "application/json"}] + end + + test "adds the port when it is not the scheme default" do + assert [{"host", "public.example:8443"}] = + Client.put_host_header([], %{@pinned | port: 8443}) + end + + test "works on the map form" do + assert %{"host" => "public.example"} = + Client.put_host_header(%{"HOST" => "spoofed"}, @pinned) + end + end + + describe "hackney_ssl_options/1" do + test "verifies the certificate against the hostname, not the pinned address" do + options = Client.hackney_ssl_options("public.example") + + assert Keyword.get(options, :verify) == :verify_peer + assert Keyword.get(options, :server_name_indication) == ~c"public.example" + assert Keyword.has_key?(options, :cacerts) + end + end + + describe "check_no_redirects!/2" do + test "passes when the caller did not ask for redirects" do + assert :ok = Client.check_no_redirects!([], "somewhere") + assert :ok = Client.check_no_redirects!([follow_redirect: false], "somewhere") + end + + test "raises when it did" do + assert_raise ArgumentError, ~r/:follow_redirect cannot be used with somewhere/, fn -> + Client.check_no_redirects!([follow_redirect: true], "somewhere") + end + end + end +end From 43f41517ccd670398f047fa141393faa5ff94647 Mon Sep 17 00:00:00 2001 From: Luis Gustavo Date: Mon, 5 Oct 2026 09:04:50 -0300 Subject: [PATCH 3/4] Drop CNAME targets from resolved records before pinning --- lib/safeurl/safeurl.ex | 11 ++++++++++- test/safeurl_test.exs | 14 ++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/lib/safeurl/safeurl.ex b/lib/safeurl/safeurl.ex index 14fc755..467c813 100644 --- a/lib/safeurl/safeurl.ex +++ b/lib/safeurl/safeurl.ex @@ -382,13 +382,22 @@ defmodule SafeURL do end end + # A resolver following a CNAME can hand back the target name next to the + # addresses, so only the addresses are kept. defp resolve_in_dns(hostname, dns_module) do case dns_module.resolve(hostname) do - {:ok, [_ | _] = ips} -> {:ok, Enum.map(ips, &normalize/1)} + {:ok, records} when is_list(records) -> only_addresses(records) _no_address -> {:error, :unresolved_host} end end + defp only_addresses(records) do + case Enum.filter(records, &is_tuple/1) do + [] -> {:error, :unresolved_host} + ips -> {:ok, Enum.map(ips, &normalize/1)} + end + end + # An IPv4-mapped (::ffff:a.b.c.d) or IPv4-compatible (::a.b.c.d) address # reaches the IPv4 host it carries, so it is checked as that IPv4 address. # The guard leaves out the unspecified and loopback addresses, which share diff --git a/test/safeurl_test.exs b/test/safeurl_test.exs index 966e4a0..2946ed3 100644 --- a/test/safeurl_test.exs +++ b/test/safeurl_test.exs @@ -11,6 +11,10 @@ defmodule SafeURLTest do def resolve("v6-internal.example"), do: {:ok, [{192, 0, 78, 24}, {0xFD00, 0, 0, 0, 0, 0, 0, 1}]} + def resolve("cname.example"), + do: {:ok, [~c"lb.example", {192, 0, 78, 24}, {192, 0, 78, 25}, ~c"lb.example"]} + + def resolve("cname-only.example"), do: {:ok, [~c"lb.example"]} def resolve("empty.example"), do: {:ok, []} def resolve("missing.example"), do: {:error, :nxdomain} def resolve(_domain), do: {:ok, [{192, 0, 78, 24}]} @@ -171,6 +175,16 @@ defmodule SafeURLTest do SafeURL.pin("https://v6-only.example/", dns_module: TestDNSResolver) end + test "pins the first address of a host behind a CNAME" do + assert {:ok, %{url: "https://192.0.78.24/", address: {192, 0, 78, 24}}} = + SafeURL.pin("https://cname.example/", dns_module: TestDNSResolver) + end + + test "refuses a host whose CNAME resolves to no address" do + assert {:error, :unresolved_host} = + SafeURL.pin("https://cname-only.example/", dns_module: TestDNSResolver) + end + test "returns the validation error" do assert {:error, :unsafe_reserved} = SafeURL.pin("http://10.0.0.1/") assert {:error, :restricted} = SafeURL.pin("http://10.0.0.1/", detailed_error: false) From 97ac966488bd377bffccf9aac9ed8635c7d52596 Mon Sep 17 00:00:00 2001 From: Luis Gustavo Date: Mon, 5 Oct 2026 09:19:38 -0300 Subject: [PATCH 4/4] Fix the tests for elixir 1.14 and otp 24 --- test/test_helper.exs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/test/test_helper.exs b/test/test_helper.exs index 869559e..156d6ce 100644 --- a/test/test_helper.exs +++ b/test/test_helper.exs @@ -1 +1,4 @@ +# Elixir < 1.15 does not start optional dependencies. +{:ok, _apps} = Application.ensure_all_started(:hackney) + ExUnit.start()