Skip to content

fix(direct): match Python's provider error classification - #7

Merged
Goldokpa merged 2 commits into
identArk:mainfrom
Kem-Ada:fix/direct-error-classification
Sep 23, 2026
Merged

Goldokpa merged 2 commits into
identArk:mainfrom
Kem-Ada:fix/direct-error-classification

Conversation

@Kem-Ada

@Kem-Ada Kem-Ada commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

DirectGateway classified provider SDK errors far more coarsely than the Python SDK does.
The OpenAI path checked only the direct class name plus a string fallback, so a 429 without a
recognisably-named class became a generic ProviderError instead of a RateLimitError, and an
auth failure was indistinguishable from any other provider error. The Anthropic paths had no
classifier at all — just two copies of the same inline string check in _callAnthropic and
_streamAnthropic, which could only ever produce ContentPolicyError or a generic
ProviderError. An Anthropic rate limit was silently flattened into ProviderError.

This ports the full detection ladder from the Python SDK's _classify_openai_error and
_classify_anthropic_error, in the same priority order: class hierarchy first (most reliable),
then HTTP status, then error code/type, then string matching.

Behaviour changes callers will notice:

  • Anthropic rate limits now raise RateLimitError (with provider === "anthropic") instead of
    ProviderError.
  • A 401 from either provider now raises ProviderError with an "Authentication failed: …" /
    "Anthropic authentication failed: …" prefix, so it is distinguishable from a transport error.
  • RateLimitError.retry_after_seconds is now populated from the provider's retry-after hint
    instead of always being the 60 default.
  • _classifyAnthropicError is shared by the blocking and streaming paths, so a streaming
    Anthropic error classifies identically to a blocking one. (This drops the previous
    "Anthropic streaming error: …" message in favour of Python's "Anthropic API error: …" —
    Python uses one classifier for both paths.)

One structural fix came along with it: the try blocks in _callOpenAi and _callAnthropic
wrapped the whole method, including response parsing and estimateCost. Python wraps only the
API call. A TypeError from an unexpected response shape was therefore being caught and
re-thrown as ProviderError, hiding real client-side bugs behind a provider-blaming message.
The blocks are now narrowed to the provider call, matching Python. There is a regression test
for this.

No new dependencies. The OpenAI and Anthropic SDKs stay optional peer dependencies — errors are
recognised structurally (prototype-chain constructor names, status, code/type), never by
importing the SDK.

Python → TS parity mapping

Reference: identark/gateways/direct.py (Python) → src/gateways/direct.ts (TS).

Python TypeScript Notes
_classify_openai_error(exc) _classifyOpenAiError(exc) Same 7-step ladder, same order
_classify_anthropic_error(exc) _classifyAnthropicError(exc) New in TS; Anthropic had no classifier
[c.__name__ for c in type(exc).__mro__] errorClassChain(exc) Walks the prototype chain collecting constructor.name
type(exc).__name__ exc.constructor.name Used for the "ContentFilter" substring check
getattr(exc, "status_code", None) errorStatusCode(exc) Reads status (what the JS SDKs use) then status_code
getattr(exc, "retry_after", None) errorRetryAfterSeconds(exc) See "retry-after" below
getattr(exc, "code") or getattr(exc, "error_code") errorCode(exc) OpenAI path
getattr(exc, "type") or getattr(exc, "error_type") errorType(exc) Anthropic path
RateLimitError(str(exc), provider=…, retry_after_seconds=…) new RateLimitError(exc.message, retryAfter ?? 60, provider) TS arg order is (message, retryAfterSeconds, provider)
("content_filter", "content_policy_violation", "output_blocked") CONTENT_POLICY_CODES OpenAI error codes
("content_policy_violation", "content_filter") ANTHROPIC_CONTENT_POLICY_TYPES Anthropic error types
f"Authentication failed: {exc}" `Authentication failed: ${exc}` OpenAI path
f"Anthropic authentication failed: {exc}" `Anthropic authentication failed: ${exc}` Anthropic path
f"{self._provider.capitalize()} API error: {exc}" `${provider[0].toUpperCase()}${provider.slice(1)} API error: ${exc}` Pre-existing; unchanged
try: … except: self._classify_*(exc) around the API call only try { await … } catch { this._classify*(exc) } around the API call only TS previously wrapped the whole method

retry-after. This is the one place the TS is not a line-for-line translation, and it is
deliberate. Python reads getattr(exc, "retry_after", None) because the Python SDK exposes it as
an exception attribute. The JS SDKs do not — they put it on the response headers. So
errorRetryAfterSeconds checks retryAfter / retry_after first, then falls back to
headers["retry-after"], handling both a Headers instance and a plain record. Non-numeric
values (e.g. the HTTP-date form of Retry-After) and non-positive values are ignored, so the
60 default still applies. Python also only reads the hint in the class-name branch, not the
429-status branch; the TS reads it in both. Both branches default to 60, so this is
identical whenever no hint is present — see Reviewer notes.

Test plan

New file tests/direct-error-classification.test.ts, 39 tests, positive and negative:

  • OpenAI rate limits (8) — SDK class → RateLimitError; bare 429 → RateLimitError;
    default 60; retry-after from a header record; retry-after from a Headers instance;
    non-numeric retry-after falls back to 60; configured provider carried onto the error;
    negative: 503 is not a rate limit.
  • OpenAI content policy (9) — ContentFilterFinishReasonError; each of the three error
    codes (parameterised); each of the three string-fallback phrasings (parameterised);
    negative: invalid_request_error is not a policy block; ordering: a 429 whose message
    contains content_filter classifies as a rate limit, proving priority order.
  • OpenAI authentication (3) — SDK class → "Authentication failed:"; bare 401 → same;
    negative: 403 falls through to "Openai API error:".
  • OpenAI fallback (3) — unclassified error is capitalised per provider; provider name is
    the resolved one (local); regression: a malformed response yields the real TypeError,
    not a ProviderError.
  • Anthropic (12) — rate limit by class and by 429; retry-after; both content-policy
    error types (parameterised); all three string-fallback phrasings (parameterised);
    negative: unrelated error type is not a policy block; auth by class and by 401;
    unclassified fallback.
  • Streaming (4) — OpenAI stream rate limit carries retry_after_seconds; OpenAI stream
    401; Anthropic stream content policy by error type; Anthropic stream 429 classifies as
    RateLimitError rather than the generic error it produced before.

Python has no unit tests for _classify_openai_error / _classify_anthropic_error — the
classifiers are exercised only indirectly. These tests are therefore written against the Python
source ladder rather than ported from a Python test file.

Full gate, run from inside identark-js:

$ npm ci
$ npm run lint
> identark@1.0.3 lint
> eslint src --max-warnings 0

$ npm run typecheck
> identark@1.0.3 typecheck
> tsc --noEmit

$ npx vitest run
 RUN  v1.6.1 /Users/starrexshotit/opensource/kemi/identark-js

 ✓ tests/managed-tools.test.ts  (2 tests) 7ms
 ✓ tests/types.test.ts  (5 tests) 5ms
 ✓ tests/control-plane.test.ts  (17 tests) 15ms
 ✓ tests/mock-gateway.test.ts  (12 tests) 16ms
 ✓ tests/vercel-integration.test.ts  (5 tests) 8ms
 ✓ tests/direct-error-classification.test.ts  (39 tests) 26ms
 ✓ tests/direct-gateway.test.ts  (13 tests) 26ms

 Test Files  7 passed (7)
      Tests  93 passed (93)

$ ../scripts/check_scope.sh
check_scope: OK — no control-plane code touched.

Lint and typecheck are clean (no output). Tests: 93 passing / 7 files, up from the
54 / 6 baseline — +39 from this PR. No existing test was modified.

Control-plane: untouched (check_scope OK)

🤖 Generated with Claude Code

Kem-Ada and others added 2 commits September 21, 2026 16:05
DirectGateway's OpenAI classifier only looked at the error class name and a
string fallback, and the Anthropic paths had no classifier at all — two
copies of an inline string check that could only ever produce
ContentPolicyError or a generic ProviderError.

Port the full detection ladder from the Python SDK's
_classify_openai_error / _classify_anthropic_error:

- class hierarchy (prototype-chain walk, the TS analogue of __mro__)
- HTTP status: 429 -> RateLimitError, 401 -> ProviderError("Authentication
  failed: ...")
- error code in (content_filter, content_policy_violation, output_blocked)
  -> ContentPolicyError; Anthropic matches on error type instead
- string fallback, unchanged

RateLimitError now carries retry_after_seconds read from the error's
retry-after header (Headers instance or plain record) or a retryAfter
property, defaulting to 60 as Python does. Anthropic errors route through a
new _classifyAnthropicError shared by the blocking and streaming paths.

Also narrows the try blocks to wrap only the provider API call, matching
Python — response parsing no longer gets laundered into a ProviderError.

Co-Authored-By: Claude <noreply@anthropic.com>
@Goldokpa
Goldokpa merged commit b2cb1ed into identArk:main Sep 23, 2026
4 checks passed
Goldokpa added a commit that referenced this pull request Oct 9, 2026
DirectGateway's OpenAI classifier only looked at the error class name and a
string fallback, and the Anthropic paths had no classifier at all — two
copies of an inline string check that could only ever produce
ContentPolicyError or a generic ProviderError.

Port the full detection ladder from the Python SDK's
_classify_openai_error / _classify_anthropic_error:

- class hierarchy (prototype-chain walk, the TS analogue of __mro__)
- HTTP status: 429 -> RateLimitError, 401 -> ProviderError("Authentication
  failed: ...")
- error code in (content_filter, content_policy_violation, output_blocked)
  -> ContentPolicyError; Anthropic matches on error type instead
- string fallback, unchanged

RateLimitError now carries retry_after_seconds read from the error's
retry-after header (Headers instance or plain record) or a retryAfter
property, defaulting to 60 as Python does. Anthropic errors route through a
new _classifyAnthropicError shared by the blocking and streaming paths.

Also narrows the try blocks to wrap only the provider API call, matching
Python — response parsing no longer gets laundered into a ProviderError.

Co-authored-by: Kem-Ada <283358351+Kem-Ada@users.noreply.github.com>
Co-authored-by: Gold okpa <Okpagold@gmail.com>
@Goldokpa Goldokpa mentioned this pull request Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants