Skip to content

Harden the HTTP layer and close out several edge cases - #1

Open
meznaric wants to merge 10 commits into
mainfrom
improve-sdk-robustness
Open

Harden the HTTP layer and close out several edge cases#1
meznaric wants to merge 10 commits into
mainfrom
improve-sdk-robustness

Conversation

@meznaric

Copy link
Copy Markdown

Summary

  • Fix plans.update() sending a null instead of a no-op when called with no arguments
  • Gate integration tests behind a dedicated opt-in variable so a plain pytest run never hits production
  • Harden the HTTP layer: follow redirects on owned clients, don't crash on a malformed pagination URL, drop empty query sequences instead of sending ?key=, cap error bodies by actual byte length, surface top-level JSON-array error bodies, raise on conflicting api_key/credentials_provider, and make timeout=None distinguishable from an omitted timeout
  • Offload async multipart file buffering to a worker thread and reject empty submit_evidence() calls client-side
  • Tighten proxy URL validation (fullmatch instead of match, percent-encoded download path segments, ASCII-only session IDs) and fix two live API mismatches (referral earnout trailing slash, replaced-proxy country code casing)
  • Fix ProxyActivity port fields being typed as float and a decode() bug that would clobber future dataclass defaults
  • Expand resource test coverage (billing, transactions, plans, profile preferences, proxy replacements, replaced proxies, referral, notifications, subusers, proxy_config stats/status) plus connection-error retry tests for both clients, and stop leaking test clients
  • Docs, build and example cleanup (retry status codes, credentials_provider cadence, hatchling pin, Python 3.14 support)

Test plan

  • pytest (132 passed)
  • mypy src/webshare tests (strict, clean)
  • ruff check . (clean)
  • Verified the referral earnout and replaced-proxy country-code behavior against the live dev API

Vito Meznaric added 10 commits July 27, 2026 12:04
A bare plans.update(id) with no kwargs sent
{"automatic_refresh_next_at": null}, which the backend rejects with a
400 (the field has no allow_null). Every other PATCH in the SDK treats
an omitted argument as "leave this field alone" via drop_json_nulls;
plans.update() was the one place that forgot it.
They auto-enabled on WEBSHARE_API_KEY, the same variable every normal
SDK user has set, so a plain pytest run with a real key would silently
hit production. Gate them behind a dedicated WEBSHARE_INTEGRATION_TEST
opt-in instead.
- Follow redirects on owned httpx clients so an http:// base URL or a
  3xx response doesn't surface as a confusing decode error.
- same_origin() no longer raises a bare ValueError on a malformed port
  in a pagination `next` URL; treated as a non-matching origin.
- Empty sequences in query params are omitted instead of sent as
  `?key=` (which some filters read as "match empty string").
- Error bodies are capped by actual UTF-8 byte length rather than
  character count, and a top-level JSON-array error body now
  contributes its messages to the exception detail instead of being
  silently dropped.
- Raise a clear error when both api_key and credentials_provider are
  passed, instead of silently ignoring api_key.
- timeout=None is now distinguishable from an omitted timeout: passing
  it explicitly requests httpx's no-timeout behavior, while omitting it
  keeps the 60s default.
Reading file inputs for a multipart upload was a synchronous disk read
happening directly on the event loop in the async client; it now runs
via asyncio.to_thread so a large or slow file doesn't stall other
tasks. submit_evidence() with neither explanation nor files now raises
client-side instead of silently sending an empty POST body.
- Country/city regexes use fullmatch instead of match, so a trailing
  newline in an otherwise-invalid value is no longer accepted.
- proxy_list_download_path now validates and percent-encodes
  country_codes/proxy_protocol so a crafted value can't inject extra
  path segments.
- session_id validation rejects non-ASCII digit characters that
  str.isdigit() alone accepts.
- referral.list_earnouts() now hits /earnout/ with a trailing slash;
  the un-slashed path 301s on the live API, which the SDK doesn't
  follow into a paginated list decode.
- replaced_proxies.download() now uppercases country codes like its
  proxies.download sibling for consistent behavior.
ProxyActivity.port/proxy_port/listen_port decoded to floats (8080.0)
because they were typed float instead of int, unlike every other port
field in the SDK.

decode() also passed every missing wire field as an explicit None
keyword argument, even fields with a dataclass default. That's
harmless today since no model field declares one, but it would
silently override any default added in the future. Missing fields are
now omitted from the constructor call when the field has a default (or
default_factory), so the model's own default applies.
- README: retries cover {408,429,500,502,503,504}, not a blanket "5xx";
  credentials_provider is called once per attempt (so it's re-invoked
  on retries), not once per request; document explicit timeout=None.
- Pin hatchling>=1.27 in [build-system] for PEP 639 license metadata.
- Add Python 3.14 to the CI matrix and package classifiers.
- proxy_replacements.create() docstring now references the public
  proxy_subtype field instead of the internal pool_filter name.
- The two examples that pick the active plan now raise a clear error
  when none exists instead of a bare StopIteration.
- Drop a stale per-file ruff ignore whose comment didn't match what the
  rule actually checks (there were no violations to ignore).
Add table-driven request-shape tests for the resource methods that had
no coverage: billing, transactions, plans, profile preferences, proxy
replacements, replaced proxies, referral, notifications, subusers
create/update/delete, and proxy_config stats/status. Add connection-
error mapping and retry tests for both the sync and async clients.

Also switch scattered `client = Webshare(...)` instantiations in
test_retries.py/test_pagination.py, and the shared fixture in
test_resources.py, to context managers so the underlying httpx client
gets closed.
A few lines from the async buffering and table-driven test changes
were over the formatter's line length; ruff format wraps them.
…est clients

Transaction.id is an int, so invoices.download now takes int | str instead
of forcing callers to stringify (verified against the live API — the query
param serializes identically). Also converted the remaining bare client
instantiations in test_client.py and test_errors.py to context managers,
and added tests for conflicting credentials, explicit timeout=None, and
redirect following.

Claude-Session: https://claude.ai/code/session_018LB1vKDSPQRHAFPowUmkeR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant