Skip to content

fix(streaming): preserve routed byte output and enforce builtin budgets - #2749

Merged
chaliy merged 5 commits into
mainfrom
fix/streaming-audit-defects
Oct 19, 2026
Merged

chaliy merged 5 commits into
mainfrom
fix/streaming-audit-defects

Conversation

@chaliy

@chaliy chaliy commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Streaming builtin readers now charge the shared input budget and admit record/collector storage before growth. Live execution delivers saved stdout/stderr descriptors through the awaited output pumps, preserves raw bytes in input process substitutions, and keeps streaming utilities live through descriptor/file redirects. Background output completing at shell exit drains before execution closes, including scripts without an explicit wait.

Why

Bounded pipes did not bound growing records or account for streamed input. Saved descriptor output could disappear from live execution, <(...) decoded binary data through a shell-text boundary, and adding 2>&1 could make a final filter wait for producer completion.

Before / After

  • A long record exceeding the live-memory cap, and streamed input exceeding the aggregate byte cap, now fail with the corresponding typed resource error; the session remains reusable.
  • exec 3>&1 >out; echo hi >&3: live stdout was empty; now sinks and capture receive hi\n, subject to output caps and sink errors.
  • cat <(printf '\377\000A'): output was ef bf bd 41; now it is ff 00 41, matching Bash.
  • Gated producers through cat 2>&1 and grep first 2>&1 now deliver their first output before the producer can finish.
  • A delayed background command ending after the final script boundary now delivers stdout/stderr exactly once to live sinks and capture. Previously its output was missing unless the script explicitly waited.

Existing buffered process-substitution, utility, background-job and tool-queue limitations remain documented.

Risk

  • Medium: builtin output routing and request accounting affect pipelines and redirects. Buffered options and diagnostics use the same already-open destinations, avoiding repeated truncation.
  • Regression coverage includes redirect permutations against Bash, saved-fd ordering, closed descriptors, caps, sink failure, binary substitutions, budget rejection/recovery and final background draining.

Validation

  • Local just pre-pr and just test passed using the container's provided Linux subreaper: 3,885 core unit tests, 2,591 integration tests (four existing ignores), and 19 failpoint tests, plus workspace/doc/script checks.
  • cargo clippy --all-targets --all-features -- -D warnings passed.
  • Nine audit regressions pass. Five CLI byte/status comparisons match GNU Bash, including delayed background completion.
  • The 96-case Bash comparison benchmark reports zero errors and 100% matching output. Parallel and SQLite benchmark workloads completed.

Checklist

  • Tests added or updated
  • Backward compatibility considered

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit c090e75 Commit Preview URL

Branch Preview URL
Oct 18 2026, 11:42 PM

@chaliy
chaliy marked this pull request as ready for review October 18, 2026 23:55
@chaliy
chaliy merged commit dd82e1f into main Oct 19, 2026
53 checks passed
@chaliy
chaliy deleted the fix/streaming-audit-defects branch October 19, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant