Skip to content

sdk: the transport is toyos::transport, and no crate of its own - #972

Merged
Japabu merged 4 commits into
mainfrom
wt/toyos-transport-merge
Oct 21, 2026
Merged

Japabu merged 4 commits into
mainfrom
wt/toyos-transport-merge

Conversation

@Japabu

@Japabu Japabu commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

The owner decided on 2026-10-21 (question L) that the transport goes into the SDK as toyos::transport, and that toyos-transport is not published. That crate was in PUBLISHED, and the trusted-publishing token publish.yml uses cannot create a crate. So every publish run since #905 has been red, and the nightly release refuses (issues/a-crate-new-to-the-published-set-...).

What changes

  • The logic moves unchanged. sdk/transport/src is now sdk/toyos/src/transport.
    • The module keeps #![forbid(unsafe_code)] and its clippy forbid/deny set as inner attributes.
    • It makes no syscall and allocates nothing.
    • toyos::net::stream is still its syscall adapter, now importing crate::transport.
    • There is no re-export, shim or helper crate.
  • Callers.
    • blockring depends on toyos for the rings.
    • netstack and its node drop the second dependency.
    • Every toyos_transport:: path is now toyos::transport::.
  • Tests and controls.
    • The loom models are now sdk/toyos/tests/transport_loom.rs.
    • Their controls are toyos features named transport-*, beside the SDK's existing log-ring-*. The bare names (no-clamp, head-only) would be ambiguous in the SDK's manifest.
    • CONTROLS runs them under package toyos, and the unit-test verdicts are now transport::…::tests::….
  • The controls gate. every_model_control_is_run read only host members' manifests, and the SDK is a guest library, so it saw none of the moved controls (red in round 1). It now also reads hostws::GUEST_LIBRARIES, which a host tests too, and NOT_A_CONTROL gains those crates' ttf and host-grant.
  • Clippy.
    • The old crate's targets were linted with --all-targets -D warnings. The workspace shapes exclude the SDK and lint its library only as a dependency of their members.
    • A new shape, -p toyos --all-targets, lints the transport's unit tests and loom models again.
    • The SDK's tests had two unrelated findings, which I fixed so that shape could be clean: byte_char_slices in log/stdio.rs and chunks_exact_to_as_chunks in relocate/tests.rs.
  • Removed:
    • the workspace member
    • the PUBLISHED entry
    • the std sources, manifests and [patch] for the crate in build/sysroot.rs (plus its tests and toolchain.rs's fixture)
    • the rustc-dep-of-std hop
    • its entries in three lockfiles
  • Fork. rust/library/Cargo.lock drops toyos-transport and toyos's edge to it in 11e31a52778. It is pushed fast-forward to the fork's main, and the rust gitlink names it. This is the lockfile cargo itself writes: starting from the old lockfile with the build's [patch], cargo metadata writes a file byte-identical to the edit (fork-lock-oracle.log). Both image builds left it unchanged.
  • Records and docs.
    • PUBLISHED's doc says a crate joins only once crates.io holds it under publish.yml.
    • The publish record stays open. Its exit is now only green publish and release runs after this lands. Every crate in PUBLISHED is in the crates.io index today, and toyos-transport answers 404.
    • The other records that name the crate, its path or its controls are updated: compile-fail, loom Arc, netstack pipes, transport heads and wifi.

Checks

check command tree exit log
merged with main (#962): host suite cargo run -- --ci host bcd97ad 0 ci-host.log
merged: x86-64 and AArch64 images cargo run -- --build-only --arch <a> bcd97ad 0, 0 x86_64, aarch64
merged: blockring in a guest cargo test --test toyos-build -- nvme_disk_boots (QEMU x86-64) bcd97ad 0 guest.log
host suite, 87 steps (all 12 transport controls reach their verdicts) cargo run -- --ci host eddc6dc 0 ci-host.log
same, before the clippy shape cargo run -- --ci host a6d4695 0 ci-host.log
first run: the gate's red cargo run -- --ci host pre-a6d4695c6 1 ci-host-red-1.log
x86-64 and AArch64 images cargo run -- --build-only --arch <a> a6d4695 0, 0 x86_64, aarch64
blockring in a guest cargo test --test toyos-build -- nvme_disk_boots (QEMU x86-64) a6d4695 0 guest.log
clippy cargo run -- --clippy eddc6dc 0 clippy.log

std's dep-info for x86_64-unknown-toyos names six files under sdk/toyos/src/transport/ and none under sdk/transport.

The branch merges main at 7bfdb92. The only conflict was blockring's crate header, which #962 rewrote; it keeps main's text with the new path. The merge changes no transport, SDK net, std or fork file.

Negative controls. Each was applied as a checked patch, built, run and restored.

  • forbid.sh:
    • #[allow(unsafe_code)] with an unsafe block inside the module fails with E0453 (exit 101).
    • The same block elsewhere in the SDK compiles (exit 0).
    • An unchecked + in the module reds cargo clippy -p toyos (exit 101).
  • controls-gate.sh: dropping transport-head-only's row reds every_model_control_is_run.
  • clippy-shape.sh: a clone_on_copy in the transport's unit tests makes --clippy exit 1.
  • e0308.sh: this re-measures the compile-fail record's row under the new path. E0308 in place of E0080 still passes cargo test -p toyos --doc, as the record says.

std streams: nothing runs them off metal. stream_handles runs only in the shared metal boot. The QEMU suite refuses it as a filter (log). Netstack's server end of a session is host-tested in toyos-net-node within the host suite.

T14: stream_handles passed at a6d4695 (the orchestrator's run). The merge changes no transport code, so it was not rerun. Its request was one testcases boot.

What the nightly release checks now

release::sdk_at_tip plans PUBLISHED: toyos-abi, toyos-keymap, toyos-font, toyos, toyos-window and toyos-osrelease. It refuses if any of them is owed at main's tip. Every crate in that list exists on crates.io, so the publish run on this merge can put up toyos (its key moves with the transport's sources) and its dependents. After that, release can name an sdk- alias again. The toolchain it packs no longer reads sdk/transport.

Risks

  • Not yet shown green: no publish run can succeed before landing. The packaging risk is mitigated: main's red publish run 39569722210 packaged and verified toyos-transport, with the same loom = "0.7" dev-dependency and the same loom test file (then tests/loom.rs). It failed only at the 403 upload (log lines 08:31:24-25Z). That run packaged toyos-transport's manifest, not the new toyos manifest, so the packaging of this exact manifest is still unshown.
  • Coupling: blockring now compiles the SDK and, transitively, toyos-abi. The kernel does not depend on blockring.
  • Metal: stream_handles ran at a6d4695, before the merge; the merge touches no transport code.

🤖 Generated with Claude Code

Japabu and others added 2 commits October 21, 2026 10:24
The owner decided (2026-10-21) that the transport belongs in the SDK and is
not published as a separate crate. `toyos-transport` was in `PUBLISHED`, and
crates.io's trusted-publishing token cannot create a crate, so every publish
run since #905 has been red and the nightly release refuses.

- `sdk/transport/src` is now `sdk/toyos/src/transport`, unchanged in logic.
  The module keeps `#![forbid(unsafe_code)]` and its clippy forbids as inner
  attributes, makes no syscall and allocates nothing, and is host-tested
  through `toyos`'s library tests. `toyos::net::stream` stays its syscall
  adapter.
- The loom models are `sdk/toyos/tests/transport_loom.rs`. Their controls are
  `toyos` features, named `transport-*` beside the SDK's `log-ring-*`, and
  `build/ci.rs`'s `CONTROLS` runs each under package `toyos`.
- `every_model_control_is_run` now reads the guest libraries' manifests as
  well as the host members', since a host tests both and `toyos` now declares
  controls. A clippy shape lints `transport_loom`; the workspace shapes already
  lint `toyos`'s library as a dependency.
- blockring depends on `toyos`; netstack and its node drop the extra
  dependency.
- The crate's workspace membership, `PUBLISHED` entry, std sources,
  manifests and `[patch]` in `build/sysroot.rs` are removed. Three lockfiles
  drop the package, and the Rust fork's `library/Cargo.lock` drops it in fork
  commit 11e31a52778, which the `rust` gitlink now names.
- `PUBLISHED`'s doc comment says a crate joins only once crates.io holds it.
  The record of the red publish runs now waits only for green runs after this
  lands. The issue records naming the crate, its path or its controls are
  updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The transport's unit tests and loom models were linted as `toyos-transport`'s
targets with warnings denied. As `toyos`'s, no shape reached them: the
workspace shapes exclude the SDK and lint its library only as a dependency.
`-p toyos --all-targets` takes that shape's place for `transport_loom` alone.
The SDK's tests had two findings, both fixed: an array of byte literals
(`byte_char_slices`) and `chunks_exact_mut` with a constant size
(`chunks_exact_to_as_chunks`).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator, T14 at a6d4695 (the image's commit; later commits are clippy and SDK test code only; one testcases boot, image hash verified): stream_handles PASS.

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #972 at eddc6dc (round 1)

Architecture

The PR does what owner decision L asks. The transport is now toyos::transport: the code is moved unchanged apart from paths, the transport- feature prefix and pub(crate)→pub(super). There is no re-export of the old crate, no shim and no helper crate. The separate crate's workspace membership, PUBLISHED entry, sysroot sources and manifests, std [patch], rustc-dep-of-std hop and three lockfile entries are all gone. git grep for toyos_transport, toyos-transport and sdk/transport finds nothing outside issues/. The only hits there are a historical evidence citation pinned to 6d2e78400, which is correct, and the publish record's history.

The transport stays separate from the syscall adapters. Its files import only core and super::, the header states the no-syscall contract, and net::stream remains the adapter. The separation now holds by convention and review rather than by the dependency graph. That is acceptable under the owner's choice. forbid(unsafe_code) is the module's inner attribute, and every submodule (arena, inflight, queue, session, stream) is declared inside it with no #[path], so it covers every moved item. forbid.sh shows E0453 inside the module and a clean build outside it.

Layering: blockring now compiles the SDK and toyos-abi. Every consumer is already userland (diskserver, fileserver, supervisor, and on current main also blockserve and nvme), and none of them reaches the kernel. kernel/**/Cargo.toml names no blockring at the PR head or at main b580e0078. #962 added toyos-scsi to the kernel, and toyos-scsi has no dependencies. blockring stays forbid(unsafe_code) and uses only toyos::transport. This follows from the owner's decision and is not a problem.

Fork pin: 11e31a52778 is a fast-forward of e1c5e93d3 and is the fork's origin/main. Its only change is 8 deletions in library/Cargo.lock, and the fork has no remaining reference to the crate. fork-lock-oracle.log shows that cargo's own resolution matches the edit. The pin is coherent.

Net size is +218/−246, almost all moved lines. Production behaviour is unchanged. The test code is the same files under new names.

Findings

BLOCKER 1. The branch conflicts with main. lib/blockring/src/lib.rs conflicts with #962 (b580e0078), found with git merge-tree origin/main eddc6dc9c. #962 rewrote the doc lines that this PR also edits, around lines 28–39. #962 added no new toyos_transport users: its disk.rs, blockserve and nvme reach the transport only through toyos_blockring. The rebase should therefore be mechanical. The merged tree has never been built, though, and #962 rewrote blockring, diskserver and the image. After rebasing, re-run cargo run -- --ci host and both --build-only images on the new head. The guest run of nvme_disk_boots at a6d4695c6 does not cover that tree.

BLOCKER 2. Two comments on the controls gate's inputs are now stale. Both are one-line fixes.

  • build/build.rs:2727-2732: NOT_A_CONTROL's doc lists every entry with its reason and calls them features "of a host workspace member". This PR adds ttf (font) and host-grant (pci-claim), which are guest-library features. The doc omits both, so it no longer describes the list. Add both with their one-word reasons, or shorten the doc so it no longer enumerates the entries.
  • build/hostws.rs:86-89: the doc says the guest libraries are separate because their "toyos is not clippy-clean on a host". This PR adds cargo clippy -p toyos --all-targets -D warnings as a passing shape, and fixed the two findings that made it fail. It also makes GUEST_LIBRARIES pub(crate) for the gate. The stated reason is now false; delete it or replace it with the real one.

For the orchestrator: two corrections to the PR description (not blockers).

  • "Negative controls → forbid.sh: An unchecked + in the module reds clippy as a dependency of blockring." The script runs cargo clippy -p toyos, which lints the SDK as the selected package, not as blockring's dependency. The same applies to "lint its library only as a dependency … I checked this with a mutation". The conclusion still holds: the new -p toyos --all-targets shape lints the library's non-test target with the module's forbids, and clippy.log shows Checking toyos under the $CONTROLS shape. Only the description of what was measured is wrong.
  • Risks, "cargo publish of that manifest is untested": main's latest red publish run, 39569722210 at b580e0078, contains better evidence. It packaged and verified toyos-transport, which carries the same [dev-dependencies] loom = "0.7" and a tests/loom.rs, with Verifying … Finished. It failed only at upload with 403 (crate creation). Verification builds the library only, and a registry dev-dependency survives packaging. Moving the same dev-dependency and test file into toyos therefore carries no packaging hazard I can find. A green run after landing remains the publish record's exit, as the record says.

SUGGESTION. sdk/toyos/src/transport/mod.rs:19: re-wrapping the paragraph left one 127-column line ("…[Inflight] table. An end keeps its own cursor locally and only stores the shared one,"). Wrap it like its neighbours.

Questions you asked

  • The control and loom tests. Every renamed feature arm is consistent: no bare old names remain in cfg/cfg!, and the only feature names left are transport-*, log-ring-* and rustc-dep-of-std. Round 2's ci-host.log, run on eddc6dc9c's exact tree with exit 0, shows all 12 rows reaching their verdicts. The unit-test controls name transport::stream::tests::… and transport::inflight::tests::…. transport_loom runs green, 19 tests, in the guest-libraries step. The six Place doc-tests, four of them compile_fail, run under Doc-tests toyos. Dropping the transport-head-only row reds every_model_control_is_run (controls-gate.sh). Each control still reverts the same decision as before.
  • Host tests. The module needs no syscalls on the host: cargo test -p toyos links none, and its tests ran in the host suite.
  • The gate. Widening declared_model_controls to GUEST_LIBRARIES is correct and smaller than listing model crates. It also pulls the SDK's log-ring-* features into the check; they match kernel-loom's rows by name.
  • Clippy coverage. The tests and loom models are linted again with -D warnings (clippy-shape.sh reds on a clone_on_copy in the unit tests). The control arms in the library are linted under $CONTROLS as a dependency. The old crate's [lints.rust] warnings = "deny" is gone, so a transport warning now fails only under clippy. Since --ci host runs clippy, that changes nothing in practice.
  • Records. PUBLISHED now names exactly the six crates the crates.io index holds. I rechecked this today: all six return 200 and toyos-transport returns 404. The publish record's narrowed exit and orchestrator owner are accurate. The compile-fail, loom-Arc, netstack-pipes, transport-heads and wifi edits are path and name updates only, and they are accurate.
  • Image evidence reused at eddc6dc. The image evidence from a6d4695c6 still applies to eddc6dc9c: I checked, and eddc6dc9c changes only #[cfg(test)] code and build/clippy.rs. The rebase in Blocker 1 makes it obsolete, though.
  • Metal. stream_handles on the T14 is fine to finish. Nothing in this change makes it necessary: the stream code is the same code under a new path, and a mistyped feature name would only fall back to the production arm. Compilation together with the controls' reds establishes the equivalence.

SEND BACK. Both blockers are small: rebase onto main and re-run the host suite and image builds, then fix the two stale comments.

Japabu and others added 2 commits October 21, 2026 10:51
#962 rewrote blockring's crate header; it keeps its text with `toyos::transport` for
`toyos-transport`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`NOT_A_CONTROL`'s doc names the guest libraries' `ttf` and `host-grant`;
`GUEST_LIBRARIES` no longer says the SDK is not clippy-clean, which the
`-p toyos --all-targets` shape disproves; the transport header's paragraph is
wrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Both send-backs are addressed. Head: bcd97ad.

(1) Merged onto main. 7bfdb92 merges origin/main (b580e00, #962).

(2) Stale comments, fixed in bcd97ad.

  • NOT_A_CONTROL's doc now names toyos-font's ttf and toyos-pci-claim's host-grant, and says "host-tested member".
  • GUEST_LIBRARIES no longer claims the SDK is not clippy-clean.
  • I wrapped the transport header's long line. That paragraph had been fused onto one line by my earlier reflow.

PR body.

  • The forbid mutation is now described as cargo clippy -p toyos, exit 101.
  • The publish risk now cites run 39569722210. That run packaged and verified toyos-transport, with the same loom = "0.7" dev-dependency and the same loom test file (then tests/loom.rs). It failed only at the 403 upload. The manifest it packaged was toyos-transport's, not the new toyos one, so I left that part stated as unshown.

@Japabu

Japabu commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator: both review findings resolved at bcd97ad (checked the fix diff 7bfdb92..bcd97ad: the two comments corrected, the header re-wrapped; the merge of main at 7bfdb92 conflicts only in blockring's header comment). Rebuilt on the merged tree: --ci host 87 green, both images, nvme_disk_boots (evidence e3b5815c3, round3). T14 stream_handles PASS at a6d4695 stands: the merge touches no transport, SDK net, std or fork code. Reviewed: LAND.

@Japabu
Japabu marked this pull request as ready for review October 21, 2026 09:06
@Japabu
Japabu added this pull request to the merge queue Oct 21, 2026
Merged via the queue into main with commit e62ebc0 Oct 21, 2026
8 checks passed
@Japabu
Japabu deleted the wt/toyos-transport-merge branch October 21, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant