Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion skills/trace-debug/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ zenrows trace export <run-id> # JSON for sharing
it. A 429 ("Rate limited") keeps its advice with or without `--retries`: wait
a few seconds and retry; if it persists, lower concurrency.
- `REQUEST_TIMEOUT` → the CLI stopped waiting; the API was reached. Raise
`--timeout` (default 120000ms, above the API's own 90s budget), or drop
`--timeout` (default 200000ms, above the API's own 180s budget), or drop
`--wait-for` so the request finishes inside that budget and the API returns
its own error. Not a connectivity problem — do not chase the network.
- `BACKEND_UNAVAILABLE` → a genuine transport failure (DNS/TCP/TLS). Check
Expand Down
2 changes: 1 addition & 1 deletion src/cli/commands/fetch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ export const fetch_: Command = {
" --output <fmt> html (default) | markdown | text | pdf",
" --screenshot capture an above-the-fold screenshot",
" --out <file> write the response body to a file",
" --timeout <ms> client-side timeout (default 120000; above the API's own 90s budget)",
" --timeout <ms> client-side timeout (default 200000; above the API's own 180s budget)",
" --retries <n> retry 429/502/503/504 and failed connects up to n times (default 0)",
" --no-signup do not auto-create a Free plan account if no key exists",
" --json print a structured result",
Expand Down
8 changes: 4 additions & 4 deletions src/core/http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ export interface ScraperParams {

/**
* Client-side timeout for a Fetch and Extract request, deliberately ABOVE the
* gateway's own 90s request budget.
* gateway's own request budget: a Fetch request can take up to 3 minutes.
*
* The two must not be equal. When they were both 90s, any request that used
* the full server budget became a race between our own abort and the API's
Expand All @@ -63,10 +63,10 @@ export interface ScraperParams {
* client must outlive the server budget so the API always gets the chance to
* answer for itself.
*/
export const DEFAULT_TIMEOUT_MS = 120_000;
export const DEFAULT_TIMEOUT_MS = 200_000;

/** The gateway's own request budget. Kept here only to justify the default above. */
export const SERVER_BUDGET_MS = 90_000;
export const SERVER_BUDGET_MS = 180_000;

/** No retry starts later than this after the first request. */
const RETRY_BUDGET_MS = 90_000;
Expand Down Expand Up @@ -459,7 +459,7 @@ export function extractDomainNotPrepared(url: string, body: string): ToolkitErro
* The CLI gave up waiting before the API answered. This is NOT unreachability:
* the connection was established and the gateway was still working, so telling
* the operator to check connectivity would send them at the wrong problem. The
* elapsed time is included so the 90s server budget is visible in the output.
* elapsed time is included so the server budget is visible in the output.
*/
export function requestTimeout(url: string, timeoutMs: number, elapsedMs: number): ToolkitError {
const suggestedMs = Math.max(timeoutMs + 60_000, SERVER_BUDGET_MS + 60_000);
Expand Down
Loading