Skip to content

Verify TLS with the OS trust store and add tls-check diagnostics - #7

Merged
Fazmin merged 4 commits into
Fazmin:mainfrom
pkilar:feat/tls-platform-verifier
Nov 1, 2026
Merged

Fazmin merged 4 commits into
Fazmin:mainfrom
pkilar:feat/tls-platform-verifier

Conversation

@pkilar

@pkilar pkilar commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Why

reqwest's rustls-tls feature only trusts the compiled-in webpki-roots. Behind a TLS-inspecting proxy, every lookup fails with invalid peer certificate: UnknownIssuer, even when the proxy's CA is installed in the operating system and curl and browsers work. There was also no way to tell from the app which CA was missing or whether a proxy was in the path.

Approach

  • Platform verifier. Certificates are now checked by rustls-platform-verifier, so the app trusts the same store as the rest of the machine:

    • macOS: the keychains (Security.framework).
    • Windows: the certificate store (CryptoAPI).
    • Linux: the system CA bundle, with SSL_CERT_FILE / SSL_CERT_DIR as overrides.

    This avoids shipping our own root list or a CA setting that users would have to keep in sync with the OS.

  • One crypto provider. rustls is pulled in with ring only, matching reqwest, so there is exactly one crypto provider and ClientConfig::builder() cannot panic over an ambiguous default.

  • Fallback. If the platform verifier cannot be set up (e.g. a Linux box with no CA bundle at all), client_builder() falls back to reqwest's built-in roots instead of failing every request, and prints the reason once on stderr (pointing at tls-check) so it is visible without --debug.

How it works

  • cve_core::tls (new module)
    • client_builder() is now the single place a reqwest::ClientBuilder is made. These clients now use it:
      • Clients (all upstream sources)
      • the source probe
      • the Ask agent's chat client
      • the model/runtime downloader in llm.rs
    • The only remaining plain reqwest::Client::new() is the loopback http://127.0.0.1 health poll for the local model server. It involves no TLS.
    • diagnose(host) makes a HEAD / request with the same proxy and verifier rules. A recording verifier captures the chain the server presented and the platform verifier's decision, without changing that decision. The report gives a hint for the common failures:
      • unknown issuer: names the CA to install, with the RHEL/Debian commands;
      • chain trusted but the request still fails: points at proxy or firewall policy;
      • no certificate at all: the connection failed before TLS.
  • cve-intel tls-check [host…] [--json]
    • For each host (default: NVD, EPSS, CISA, OSV, GitHub) it prints the proxy in use (credentials masked), the CA store read, each certificate's subject, issuer and SHA-256, and the verdict.
    • It exits non-zero if any host fails.
    • User-supplied hosts go through validators::validate_domain.
  • Global --debug flag
    • Installs a small stderr logger for the rustls, rustls_platform_verifier, reqwest and hyper_util targets. Stdout stays clean for --json and MCP stdio.
    • At these levels those crates log connection setup only, not bodies or headers.
  • Docs. The README has a new "HTTPS, proxies and private CAs" section and the command reference. .env.example documents SSL_CERT_FILE, SSL_CERT_DIR, HTTPS_PROXY and NO_PROXY.

Type of change

  • Bug fix (lookups fail behind TLS-inspecting proxies whose CA is in the OS store)
  • Dependency update (rustls 0.23 with ring, rustls-platform-verifier 0.6, log 0.4)
  • Documentation update

Testing

  • Added unit tests in tls.rs: the platform verifier and client build, proxy credential redaction, NO_PROXY matching, hint selection, and X.509 subject/issuer parsing from a fixed test certificate.
  • cargo fmt --all --check, cargo clippy -p cve-core -p cve-server --all-targets -- -D warnings, cargo test -p cve-core -p cve-server (266 passed) and cargo check -p cve-intel-desktop all pass locally on Linux.
  • Tested through the CLI:
    • cve-intel tls-check services.nvd.nist.gov api.osv.dev verifies both chains against the system store and exits 0.
    • --debug tls-check api.first.org logs the verifier loading the system roots and the connection setup to stderr.
    • cve-intel cvss … still prints 9.8.
  • N/A private-IP blocking: tls-check is a CLI-only diagnostic the local user runs. It is not exposed over MCP, REST or the agent.

Checklist

  • Business logic lives in cve-core (tls.rs); the CLI only formats the report.
  • Input validation uses the existing validator utilities (validate_domain).
  • No API keys or sensitive data appear in logs, cache, or responses. Proxy credentials are redacted (with a test), and --debug is limited to connection-setup targets.
  • No change to what is sent to a cloud AI provider.
  • No new tool or data source; the new CLI command is documented in README.md.

reqwest's rustls-tls feature only trusts the compiled-in webpki-roots, so
a TLS-inspecting proxy whose CA is installed in the OS failed with
"invalid peer certificate: UnknownIssuer".

Every HTTP client now gets its rustls config from tls::client_builder(),
built with rustls-platform-verifier: the macOS keychains, the Windows
certificate store, or the Linux system bundle (SSL_CERT_FILE/SSL_CERT_DIR
override it).

`cve-intel tls-check [host...]` shows the proxy used (credentials
masked), the CA store read, the chain each host presents and the
verifier's decision, with a hint. The global `--debug` flag logs rustls,
verifier and connection details to stderr.
client_builder() reported a failed platform verifier through log::warn!,
but no logger is installed unless --debug is passed, so a machine with
no usable CA store silently fell back to the built-in roots. Print the
reason once per process on stderr instead, pointing at tls-check.

That was cve-core's only log call, so drop the log dependency there and
the cve_core target from the --debug filter. Also fix the Cargo.toml
comment to name tls::client_builder.

@Fazmin Fazmin left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, it's a really solid piece of work. Trusting the OS store is the right call for people behind TLS-inspecting proxies, and tls-check is going to save a lot of back-and-forth on support issues. I like that the recording verifier only lives inside diagnose() and hands the platform verifier's decision back unchanged. Routing every client through client_builder() is a nice cleanup too.

A few small things I'd like fixed before merging, each only a line or two:

1. client_config() should pick ring explicitly (tls.rs ~L22). ClientConfig::builder() only works while exactly one rustls provider is compiled in. That's true today, but rustls' default features (and hyper-rustls, tokio-rustls, or reqwest 0.13's rustls feature) bring in aws-lc-rs. If a future dependency or Tauri plugin pulls one of those in, this panics inside Clients::new and the app crashes at start-up. client_config_recording() already uses builder_with_provider(ring), so using the same thing here makes the "exactly one provider" guarantee hold in the code and not just in Cargo.lock.

2. redact_userinfo can leak a proxy password (tls.rs ~L245). It cuts the authority at the first / before looking for @, so with HTTPS_PROXY=http://alice:pa/ss@proxy:3128 the password comes out unmasked in both the text and the --json output. That's exactly the kind of output people paste into issues. Splitting on the last @ across the whole remainder (or parsing with url and masking everything if parsing fails) would fix it. A test for it would be great.

3. The hint is wrong when the verifier can't be set up (tls.rs ~L100 and ~L355). On a Linux box with no CA bundle, the fallback warning tells people to run tls-check. diagnose then fails with "could not set up the platform verifier…", the chain is empty, and the hint says the connection "failed before TLS (DNS, firewall…)". That sends them the wrong way. A specific hint for this case ("no CA certificates were found; install ca-certificates or set SSL_CERT_FILE") would close the loop.

And some smaller notes, not blocking:

  • --debug never shows anything from rustls: it's built without its logging feature, so it has no log dependency. Either add "logging" to the features or drop rustls from the README and PR description.
  • The UnknownIssuer hint prints the Linux commands (update-ca-trust, SSL_CERT_FILE) on macOS and Windows too, where they don't apply. I'm also not sure macOS reports a proxy's untrusted root as UnknownIssuer at all, rather than as a generic "not trusted" error. If you have a Mac handy it might be worth a quick check.
  • Certificate subject and issuer are printed raw in the text output, so a hostile server could slip terminal escape sequences into it. Stripping control characters would be enough.
  • diagnose is pub but only safe as a CLI tool, since validate_domain doesn't resolve names or block private addresses. A doc comment saying it shouldn't be exposed over MCP or REST as it is would help whoever touches it next.
  • trust_store_paths() has no branch for targets other than Linux, macOS and Windows, so it won't build on Android. That isn't a target for us, so this is just an FYI.

Once those three are in, I'm happy to merge this. Then #8 can be rebased on top. Thanks again!

- Name the ring provider in client_config(), shared with tls-check, so
  a second rustls provider pulled in by a dependency cannot panic at
  start-up.
- Mask everything before the last '@' in proxy URLs, so a password
  with an unescaped '/', '?' or '#' is not printed.
- Give a specific hint when the platform verifier cannot start (no CA
  certificates) instead of blaming DNS or the firewall.
- Make the "add the CA" advice match the OS, treat macOS's
  errSecNotTrusted like UnknownIssuer, and hint on other rejections.
- Show control and bidi characters in server-supplied text as \u{..}.
- Enable rustls' logging feature so --debug shows handshake details.
- Document that diagnose() must stay a local CLI tool; fix the Android
  cfg in trust_store_paths().
@pkilar

pkilar commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review! Everything is addressed in 8240bcb.

Blocking

  1. client_config() now uses builder_with_provider(ring). It shares a provider() helper with client_config_recording(), so both pick the provider the same way.
  2. redact_userinfo now masks everything before the last @, so http://alice:pa/ss@proxy:3128 becomes http://***@proxy:3128. Tests cover /, ?, #, @ and :// in the password. The trade-off is that an @ in a proxy path now gets over-masked, which seemed the safer side to err on.
  3. When the verifier can't be set up, tls-check now says no CA certificates could be loaded and suggests installing ca-certificates or setting SSL_CERT_FILE/SSL_CERT_DIR, instead of blaming DNS or the firewall. I checked it by pointing both variables at a missing path.

Notes

  • --debug: I enabled rustls' logging feature, so it now logs handshake details (cipher suite, resumption, ALPN). The only lockfile change is log added to rustls's dependencies.
  • The CA advice now depends on the OS: Keychain / security add-trusted-cert on macOS, certmgr / certutil on Windows, and the Linux commands elsewhere.
  • On macOS you were right. From rustls-platform-verifier's code, an untrusted root comes back as errSecNotTrusted (-67843) inside a generic Other error, not as UnknownIssuer. The hint now matches that code too, and any other rejection gets a general hint instead of none. I haven't tried it on a real Mac yet.
  • Subject, issuer, the verifier error and the verdict now show control characters and bidi overrides as \u{..}.
  • diagnose has a # Security doc comment saying it must stay a local CLI tool.
  • I also fixed the Android cfg in trust_store_paths().

I'll rebase #8 once this is merged.

@Fazmin

Fazmin commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Thanks so much, Paul! This all looks great. I really appreciate how quickly you turned it around, and the extra care on the macOS hint and the terminal escapes.

@Fazmin

Fazmin commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Hey Paul, one tiny thing before I merge, sorry! CI is all green apart from the dependency audit. rustls-platform-verifier pulls in webpki-root-certs 1.0.6, which picked up an advisory a couple of weeks ago (RUSTSEC-2026-0383): its bundled roots still trust a CA root whose leaked intermediates are now distrusted. We're not actually affected, since the verifier only uses that crate on wasm, but the audit checks the whole lockfile.

Could you run cargo update -p webpki-root-certs (it goes to 1.0.11, nothing else changes) and push? I'll merge as soon as it's through. Thanks again!

rustls-platform-verifier pulls it in for wasm builds only, so native
builds were not affected, but cargo audit scans the whole lockfile.
@pkilar

pkilar commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

Hi Fazmin,
it's pushed.
Thanks for this tool, it's simply awesome and saves me tons of work :)
I have another PR I'm working on that implements OAuth for the CLI version, it should be ready early next week.

Thanks!

@Fazmin
Fazmin merged commit ca0ebf6 into Fazmin:main Nov 1, 2026
5 checks passed
@Fazmin

Fazmin commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Merged! Thanks so much, Paul. And thank you for the kind words, that really made my day :) Looking forward to the rebased #8 and the OAuth PR next week!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants