[pkilar]

Builds on #7. It uses tls::client_builder(), so this branch is current main plus #7's two commits plus three new commits. Please review only the last three: feat: add Remote AI with live model lists and custom endpoints, fix: harden Remote AI key binding, error text and retry memory and fix: fail closed when the Remote AI HTTP client cannot be built. Once #7 merges, this branch will be rebased so only those remain.

Why

Ask's cloud providers are three fixed endpoints (OpenAI, Anthropic, Gemini), so there is no way to point Ask at an OpenAI-compatible gateway such as a LiteLLM proxy, vLLM or Azure OpenAI. The model choice is a hand-typed id, so users have to guess which models their key can use. Newer models also reject parameters Ask always sends:

• GPT-5 and the o-series accept only the default temperature.
• Newer Claude models deprecate temperature.
• Some servers want max_completion_tokens instead of max_tokens.

When that happens, Ask fails with a truncated JSON error. Providers can also only be configured in the desktop app's Settings.

Approach

• A fourth provider, custom. It covers any OpenAI-compatible endpoint, with a user-set base URL and an optional key. The three built-in providers keep their compiled-in URLs.
• The base URL is validated, not trusted.
  • The URL must be https://. Plain http:// is allowed only to loopback, and loopback is decided from the parsed host, not by string match.
  • Credentials, a query string or a fragment in the URL are rejected.
  • A saved key is bound to its URL: changing the URL forgets the key, so an old key is never sent to a new host.
  • A key saved while the URL comes from CUSTOM_LLM_BASE_URL is saved together with that URL. A saved key is never paired with a URL from the environment, so changing the variable cannot redirect it.
  • Remote requests don't follow redirects, and loopback endpoints bypass HTTP(S)_PROXY.
  • If the HTTP client cannot be built with those rules, the request fails. It never falls back to a default client that would follow redirects or use the proxy.
• Live model lists. Model lists are fetched from GET <base>/models with the saved key, instead of a hard-coded list.
  • Anthropic gets x-api-key.
  • For OpenAI and Gemini, embedding, speech and image models are filtered out.
  • "Other" still accepts any model id.
• Parameter fallback. Defaults are guessed per provider and model. If a request gets a 400 or 422 that names one of these parameters, it is retried once without that parameter. Once a retried request succeeds, what worked is remembered per endpoint and model for the session; a retry that fails teaches nothing. Swapping max_tokens and max_completion_tokens needs the error to say the field is unsupported, so a complaint about its value does not trigger it. Each retry is logged at the cve_core target, which --debug shows.
• Clearer errors, without the key. Errors show the provider's own message instead of raw JSON. The key is replaced with [API key] in every response body first, since some providers and gateways repeat it.

How it works

• cve-core/src/provider.rs
  • PROVIDERS gains custom, with CUSTOM_LLM_BASE_URL / CUSTOM_LLM_API_KEY as environment fallbacks.
  • validate_base_url() checks custom URLs.
  • update() saves the model, URL and key together and enforces the key-to-URL binding.
  • list_models() and parse_models() build the model picker.
• cve-core/src/agent.rs
  • ChatClient disables redirects for remote endpoints and turns off the proxy for loopback ones. A failed build is an error: Test and the model list report it, and Ask answers from the gathered lookups with a note, as it does when the model cannot be reached.
  • Quirks (default_for, adjusted_for) and the per-endpoint cache drive the parameter retry.
  • error_message() extracts the provider's message from OpenAI-, Anthropic-, Gemini- and FastAPI-style error bodies.
• cve-core/src/settings_db.rs: providers gains a base_url column. open() adds it to existing databases.
• CLI. cve-intel ai list|set|models|use|remove-key|test. Keys are never taken from argv. ai set --key reads the key from a hidden prompt, or from stdin when piped, and ai list shows only where a key comes from.
• Desktop. A Remote AI view under Configure (moved out of Settings), with a live model picker and "Save without switching". Errors stay until dismissed and can be selected and copied. A new ai_models Tauri command backs the picker. The UI still receives only key status, never key values.
• Docs. README.md documents the CLI and the custom endpoint. SECURITY.md covers where requests go now that a user can choose the host. .env.example adds the two new variables.
• log in cve-core. cve-core depends on log again, for the retry note, and the --debug filter shows the cve_core target. Verify TLS with the OS trust store and add tls-check diagnostics #7 had dropped both when they were unused.

Type of change

• AI provider / Settings change (OpenAI, Anthropic, Gemini)
• Local model / agent change
• Desktop UI change
• Documentation update

Testing

• Added 18 unit tests. Endpoints are mocked with a TcpListener on 127.0.0.1, so nothing touches the network. They cover:

  • base URL validation, and the key being bound to its URL, including a key saved while the URL came from the environment;
  • environment URL fallback, with an invalid value ignored;
  • migrating a database that lacks the base_url column;
  • model-list parsing and request shape (including x-api-key), and that a keyless custom endpoint sends no Authorization header;
  • loopback endpoints bypassing an environment proxy, and a client that cannot be built being an error;
  • extracting provider error messages, and the key never appearing in chat or model-list errors (JSON, HTML and non-JSON bodies);
  • rejected parameters being retried and remembered, a failed retry not being remembered, value complaints not swapping the token-limit field, and other rejections surfacing the provider's message.

  I confirmed each of the four tests added by the fix commits fails when its fix is removed.

• All of these pass locally on Linux:

  • cargo fmt --all --check
  • cargo clippy -p cve-core -p cve-server --all-targets -- -D warnings
  • cargo test -p cve-core -p cve-server (272 core and 16 server tests)
  • cargo check -p cve-intel-desktop
  • scripts/version.sh check

• Tested ai list and ai set custom --url … through the CLI, against a scratch CVE_INTEL_HOME:

  • Refused: http:// to a remote host (including localhost.evil.com), credentials in the URL, and a query string.
  • Saved: http://127.0.0.1:4000/v1 and an https:// URL.
  • The existing settings.db gained the base_url column.
  • ai set custom --key is refused when there is no URL anywhere. With the URL in CUSTOM_LLM_BASE_URL, the key is saved with that URL, and changing the variable afterwards does not move the key.

Checklist

• Business logic lives in cve-core (provider.rs, agent.rs). The CLI and Tauri commands only call it.
• What is sent to a remote AI provider, and where, is disclosed in README.md and SECURITY.md.
• The new environment variables are documented in .env.example.

[Fazmin]

Thanks for this one! Being able to point Ask at a LiteLLM, vLLM or Azure gateway, and pick from the models a key can actually use, is a big improvement. I can tell how much care went into the security side: binding the key to its URL, no redirects, failing closed and the redaction all hold up well. The URL validation tests are great too.

Since this builds on #7, let's get #7 merged first (I've left a few small notes there), and then this can be rebased so only your three commits are left.

Two things I'd like fixed before merging:

1. An environment key can still end up at a different host. key_for (provider.rs ~L375) falls back to CUSTOM_LLM_API_KEY whatever URL is in use, including one saved in the settings DB. On top of that, the Settings screen pre-fills the Base URL field with the env URL and always sends baseUrl, so just changing the model copies the env URL into the DB. Here's how it plays out:

• The env has CUSTOM_LLM_BASE_URL=A and CUSTOM_LLM_API_KEY=kA. I change the model in the app, and A is now saved.
• Later I move to gateway B and set the env to B and kB. The saved A still wins, so kB goes to A.

In the same way, with the env pair set, typing a new URL in the app sends the env key there without the "Change the endpoint address?" prompt, because that prompt only appears for a saved key. I think the fix is to only pair the env key with the env URL, and to not send baseUrl from the UI when it came from the env and wasn't edited.

2. The live model list doesn't load for a brand-new custom endpoint. Custom has no default model, so even with the URL and key saved it still counts as "missing". The refresh button stays disabled (p.missing && !p.model, index.html ~L3137) and the auto-load is skipped (~L3156) until you type a model id by hand. The hint also says "Save a key…" when a key is already saved. cve-intel ai models custom works fine, so it's only the UI gate.

A few smaller things, none of them blocking:

• without_key runs on the raw body before the JSON is parsed, so a key echoed back with JSON escapes (e.g. / as \/) survives into the message error_message returns. Running it on the extracted message as well would close that gap.
• Env keys skip validate_key. A very short one (say x) would then get swapped out all over a successful response body by without_key.
• Dropping temperature for every Claude model quietly changes the default for existing setups, from 0.2 to the provider's default. Since the new retry already handles the "deprecated" 400, could we keep 0.2 and let the fallback handle the models that reject it?
• I think Anthropic's /v1/models returns 20 models per page by default, so the list may be cut short without a limit (or paging).
• fold_litellm migrates a litellm provider that never shipped on main, so I think it can go.
• In cli.rs, read_hidden_line's doc comment has ended up on run_ai.
• Update derives Debug while it holds api_key. Nothing prints it today, but it'd be nice to match Stored and LlmEndpoint.

Really nice work overall. Ping me when it's rebased and I'll take another look.

[pkilar]

Thanks for the thorough review! #7 is merged, so I've rebased this onto main. It's now my three commits plus one commit for the notes below (a53ff61).

1. Env key reaching a different host. CUSTOM_LLM_API_KEY is now only paired with CUSTOM_LLM_BASE_URL. A URL saved in the app gets the saved key, or the env key only when the saved URL is the same as the env URL. On the Settings side, baseUrl is sent only when the field was edited, so changing the model no longer copies the env URL into the DB. Typing a new URL while the env pair is set now shows the "Change the endpoint address?" prompt too, saying the env key won't be used for the new address. environment_key_goes_only_to_the_environment_url walks through both of your scenarios.

One leftover case: a DB where the old UI already saved the env URL (only possible for someone who ran this branch) keeps using that URL, but now without a key, so requests fail with an auth error instead of sending the key elsewhere. Entering the current env URL in Settings once fixes it.

2. Model list for a new custom endpoint. The provider view now has can_list_models, which is true once the URL and any required key are set, with no model needed. The refresh button, the auto-load and the hint all use it, so "Save a key…" only shows when a key is actually missing. Covered by model_list_needs_url_and_key_but_no_model.

Smaller items:

• Escaped key echo: error_message runs without_key again on the decoded message, before clipping, on both the JSON and plain/HTML paths. The test covers \/ and s escapes, plus a key right at the clip point.
• Env key validation: env keys now go through validate_key. One that fails is ignored, and the status says why ("ANTHROPIC_API_KEY is set but does not look like an API key…") instead of just "missing".
• Claude temperature: back to 0.2. Models that reject it hit the retry, which remembers that for the rest of the run, so the cost is one rejected request per model per app run.
• Anthropic model list: requests ?limit=1000, the API's maximum, so I didn't add paging. An account would need more than 1000 models to be cut short.
• fold_litellm: removed, along with its test.
• read_hidden_line: the doc comment is back on the right function.
• Update Debug: hand-written now and redacts api_key, like Stored and LlmEndpoint, with a test.

I also added a sentence to README, SECURITY.md and .env.example saying the env key is only sent to the env URL. fmt, clippy, the tests and the desktop check all pass locally.

[Fazmin]

Merged! Thanks again, Paul. The fixes are spot on, and I really like how the env key is handled now. Looking forward to #9 and the OAuth one!