Skip to content

Follow public redirects when checking chat links - #225

Merged
iamwavecut merged 1 commit into
mainfrom
fix/public-crawl-redirects
Oct 10, 2026
Merged

iamwavecut merged 1 commit into
mainfrom
fix/public-crawl-redirects

Conversation

@iamwavecut

Copy link
Copy Markdown
Owner

A direct request to verify a Mozilla page failed because crawl_url rejected its ordinary HTTP 301 response. Follow up to five HTTP redirects, validating and pinning every destination again. One deadline covers DNS, all requests, and body reads. Keep private-network, credentialed-URL, proxy, binary-body, and decoded-size restrictions.

Clarify that search results alone do not mean a page was read. The SAFE probe previously acknowledged a failed crawl but then labelled the search result as a read page.

Validation: formatting and diff checks, 2466 workspace tests passed (10 ignored), all-target Clippy with warnings denied. Redirect tests cover relative locations, supported statuses, the hop limit, missing or invalid locations, and private or unsupported destinations. The same live Mozilla crawl failed with 301 before the change and returned Firefox page text after it.

@github-actions

Copy link
Copy Markdown
Warnings
⚠️ Prompt files are cache-sensitive contracts; confirm prompt shape and role markers were preserved intentionally.
Messages
📖 Danger checked 2 changed file(s).

Generated by 🚫 dangerJS against 09c1811

@iamwavecut

Copy link
Copy Markdown
Owner Author

Prompt roles, partials, rendering data, and the stable-prefix/dynamic-packet layout are unchanged. The added sentence requires a successful crawl result before calling a page read. The public prompt behavior changes intentionally; workspace prompt rendering and LLM contract tests passed.

@iamwavecut
iamwavecut merged commit 5937212 into main Oct 10, 2026
12 checks passed
@iamwavecut
iamwavecut deleted the fix/public-crawl-redirects branch October 10, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant