Skip to content

fix(llm): preserve request options across transport retries - #14

Open
VeraPyuyi wants to merge 1 commit into
AetherLabsAI:mainfrom
VeraPyuyi:fix/preserve-retry-request-options
Open

VeraPyuyi wants to merge 1 commit into
AetherLabsAI:mainfrom
VeraPyuyi:fix/preserve-retry-request-options

Conversation

@VeraPyuyi

@VeraPyuyi VeraPyuyi commented Oct 18, 2026 •

Copy link
Copy Markdown

Problem

chat() builds response_format, top_p, reasoning and provider-routing options from the role configuration. On an ordinary transport exception (429, 503, connection error, or timeout) with no pinned provider order, it then reset kwargs = {} and retried immediately. A transient outage could therefore silently remove reasoning effort, sampling, and JSON output settings from the rest of that request attempt.

The shipped Curriculum and Verifier roles use empty provider routing, so the affected path is part of the released configuration. The outer work-clock pause also treats this as successful generation.

Change

Remove the generic option-dropping branch. Every retry now sends the same complete request, consistent with the existing behavior for HTTP-200 embedded provider errors.

The four-attempt limit, backoff, error classification and LLMTransportError contract are unchanged. A provider that genuinely rejects an optional parameter still ends in the existing explicit fatal transport error; it is no longer silently converted into a different experiment. No role profiles, benchmark locks or public signatures change.

Validation

Python 3.12.15, offline fake SDK client; no model, network, VM or benchmark calls.

Check Result
New cases on unchanged llm/client.py 6 failed, 13 passed
tests/test_llm_response_errors.py after fix 19 passed
Response-error + transport-pause + reasoning-thread tests 30 passed
Durable images + eyes + mixed-action transport 34 passed
git diff --check clean

New cases parameterize 429, 503, connection failure and timeout across empty and pinned provider routing. They compare deep-copied full requests (history, exact image bytes, reasoning, top_p, response format), then confirm the successful reply and provider provenance. Fatal 400/422 exhaustion cases confirm options survive all four attempts and the error remains non-recoverable.

Windows cannot collect the official suite because Unix-only fcntl is imported at module load. With that one module excluded, the final diagnostic has 640 passed, 141 failed, and 2 skipped. Re-running with the unmodified client and the same expanded tests produced 631 passed, 150 failed, and 2 skipped. All 141 final failures also occur in that baseline; six of the removed failures are the new retry regressions, and three vary with platform/test ordering. These diagnostics contain platform-specific symlink, fcntl, shell and path assumptions. No final-only failures were observed. Linux CI is the authoritative full-suite check; no green full-suite result is claimed here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant