Skip to content

sshserver on toyos-ssh with std threads; russh, tokio and rsa are gone (SSH stage 2) - #1026

Merged
Japabu merged 6 commits into
mainfrom
wt/sshserver-on-toyos-ssh
Oct 23, 2026
Merged

Japabu merged 6 commits into
mainfrom
wt/sshserver-on-toyos-ssh

Conversation

@Japabu

@Japabu Japabu commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

system/sshserver served SSH on the russh fork over tokio. That brought rsa (RUSTSEC-2023-0071), pageant, the russh-internal bignum and ssh-key forks, and p384 0.14. p384 needed the experimental bignum backend set for every build to keep fiat-crypto out. The daemon also kept one input sender per connection, so a second channel's data reached the first channel's program. Stage 2 of issues/the-ssh-server-is-toyos-own-and-russh-is-gone.md puts it on toyos-ssh. The owner ruled on 2026-10-22: "Use our own crypto where possible; we optimize and improve until done, but we can already use it."

Result

  • One thread per connection drives toyos-ssh's sans-IO server (system/sshserver/src/connection.rs).
    • The socket's bytes, a program's output, what its stdin took and its exit all reach that thread on one queue. That thread alone touches the protocol and writes the socket.
    • Flow control is the protocol's. The client's window comes back only as a program's stdin takes the bytes, so the old 30-second input stall is gone. A pipe's reader waits until its chunk is in the client's window before it reads more.
  • It serves what it served. A shell, with CR LF translation when a terminal was asked for; exec through command.rs; and subsystem sftp over sftp.rs, on a thread of its own.
  • It adds what russh did not give:
    • several channels on one connection, keyed by toyos-ssh's ChannelId, which ends the input aliasing;
    • an authorized_keys reader in toyos-ssh, written from sshd(8)'s format: an entry with options authorizes nothing, and another key type is named and not taken;
    • a 120 s login deadline, OpenSSH's LoginGraceTime, which bounds a client that only offers keys. Before login it bounds every wait and the whole of every write, each send given only the time left, so a client that stops reading, or reads too slowly, is closed at the deadline too;
    • a cap of 10 connections that have not logged in, OpenSSH's MaxStartups. One more is closed at once.
  • Input waits as output does. The socket's reader hands over one chunk at a time and waits until the connection took it, so a client that sends faster than the connection answers waits on its socket.
  • A program ended by its channel's or connection's end is killed and reaped, so a host keeps no zombie.
  • toyos-ssh gains shell, the sftp subsystem (other subsystems are declined by name) and one pty-req before a program. Event::Authenticated names the key's fingerprint. The host key is shared by Arc.
  • Deleted.
    • From the root [patch]: russh, russh-cryptovec and tokio.
    • From Cargo.lock: rsa, pageant, internal-russh-num-bigint, internal-russh-forked-ssh-key, p384 0.14 and fiat-crypto 0.3.
    • From .cargo/config.toml: the p384_backend setting.
  • toyos-ssh moves to lib/ssh. It now has two users, the build and sshserver, which is step 5 of issues/the-tree-says-who-uses-each-thing.md.
  • Records.
    • Closed: sshserver-holds-one-channel-and-does-not-say-so, sshserver-still-speaks-rsa, and the-tokio-forks-root-manifest-patches-mio, which has no consumer left.
    • Updated to stay true: the SSH track (stage 2 landed; stage 4 is mio and socket2 alone), the crypto track's consumer rows, and the issues that named russh or tokio as the current state.

Net code change: +1,243/−889, outside fixtures, Cargo.lock and issues.

  • Production: connection.rs 423, main.rs 122 against 799 deleted, program.rs 46, and authorized_keys with its tests 148. The core's request handling is about 90.
  • Tests: the in-memory SFTP server 201, and the replay, refusal, fuzz and driver changes about 180.

Decisions

  • A program that closed its stdout and stderr and runs on is no longer ended with its connection. I'd like a decision on this one.
  • The recorded sftp session replays against toyos-ssh's scripted driver, whose SFTP server is in memory (lib/ssh/tests/common/sftp.rs). It is not sftp.rs.
    • sftp.rs answers REALPATH "." with the process's working directory, and OpenSSH builds its absolute paths from that answer. No recording made on the Mac could replay byte for byte on another host.
    • sftp.rs is covered by its own 11 tests and by the Mac end-to-end run below with OpenSSH's sftp.
  • sshserver stays an app that builds on every host. That is what lets the end-to-end run below use the Mac.

Checks and evidence

Evidence, every log masked: https://github.com/ToyOSOrg/evidence/tree/28a0e72a9e398dafb29069d2679186e54d6f1e98/pr-1026 (round 1 in r1/, round 2 in r2/)

Head 2b38b4f, on main 35e9301.

  • cargo run -- --ci host at 2b38b4f, after the rebase changed Cargo.lock: exit 0, 101 steps green (r2/ci-host.log).
  • Images, at round 1's 81e922b: aarch64 --build-only exit 0, and the x86_64 image built for the QEMU run below. Round 2 changes the write loop's portable code and a comment, both covered by --ci host's clippy and tests.
  • Host tests of the driver (system/sshserver):
    • output_goes_whole_and_in_order_through_a_narrow_window: a chunk the window takes in part waits for the rest, in order, and its reader is told only once all of it went.
    • a_client_that_floods_offers_and_never_reads_is_closed_at_the_deadline: a loopback client completes the exchange, then floods unsigned offers, each answered PK_OK, and never reads. The connection ends at its 2 s deadline and gives its place back.
    • a_write_to_a_slow_reader_ends_at_the_deadline: a sink that takes one byte a send, each send making progress, is given up at the deadline.
    • a_program_ended_is_reaped.
  • The guest check, on QEMU x86-64: stream_handles gains two parts.
    • One end on two threads: a write completes while the other thread is in a read the peer has not answered, and that read then takes what the peer publishes.
    • Piped children across threads: stdin written, stdout and stderr read and the exit waited for, each on its own thread with more than a pipe holds in flight; and a child killed from one thread while another reads its stdout, whose read then ends.
    • Run through a temporary QEMU row (r1/guest/temporary-qemu-row.patch, not committed): a discovered binary otherwise runs only in the T14's shared boot, where it stays registered.
  • OpenSSH recordings against toyos-ssh (OpenSSH_10.3p1). Each negotiated mlkem768x25519-sha256 and replays byte for byte, also cut at every byte and at every 61st:
    • ssh -tt with a shell;
    • sftp -b with a put, an ls -l, a get compared with what was put, and an rm the in-memory server refuses;
    • an ssh -M connection carrying a cat held open while an echo ran on a second channel.
  • End to end on the development Mac (r1/e2e/), at the round-1 code: the host build of sshserver on port 22, OpenSSH as a black-box client, 17 steps, all as intended:
    • exec, exit 7, and stderr apart from stdout;
    • 50 MB in, 50 MB out, and 50 MB out to a reader that pauses;
    • a missing program and a missing shell, each exit 127;
    • sftp of 20 MB, and two channels at once;
    • a dropped connection's program ended and reaped, with no zombie left;
    • a non-reading program stalling only its own channel;
    • a stranger key and a restrict key refused;
    • the cap, the 120 s deadline, and the host key kept across a restart.
  • Negative controls, each a checked patch applied, built, run and reverted:
    • Round 2 (r2/controls/): round 1's single timeout over write_all fails the slow-reader test, writing 200 bytes over 5.4 s against a 200 ms deadline.
    • Round 1 (r1/controls/):
      • dropping the rest of a partial send fails the narrow-window test;
      • giving pre-login writes an hour fails the flood test on its minute bound, because the connection never ends;
      • killing without reaping fails the reap test;
      • publishing the peer's later bytes without ringing stalls the guest check red.
    • Before (controls/):
      • an options entry that authorizes;
      • every shell reported as having a terminal;
      • any subsystem served;
      • a second program on a channel;
      • an eleventh unauthenticated place.
        Each fails its named test.
  • Not run: a login to sshserver on ToyOS. Stage 5's metal row is that, and its client is undecided. The guest check above holds the std behaviour on ToyOS that the daemon relies on.

Risks

  • No ToyOS login, as above; the guest check holds the std behaviour it depends on.
  • Programs closed under the hole above outlive their connection.
  • authorized_keys ignores the user name, as before: every login runs as the daemon.
  • issues/sshserver-reports-a-killed-program-as-exit-lost.md stays open: toyos-ssh sends no exit-signal.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q

Japabu added a commit that referenced this pull request Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review, round 1: #1026 at b6e6912 (sshserver on toyos-ssh), this PR's commits cdd3ddf and b6e6912 only

Architecture. The shape is right. The sans-IO core owns every byte a client sends before a program runs. The daemon owns the sockets, the processes, the files and the policy: the deadline, the cap and the authorized_keys path. One thread per connection is the only toucher of Server and of the socket's write side, and everything else reaches it as an Input on one queue. Flow control is the protocol's end to end: the client's window comes back only as stdin takes bytes, and each pump has one chunk in flight, acknowledged by taken.

  • These are bounded: per channel, 2 MiB of client data queued for stdin and 32 KiB of output per stream; ten channels per connection.
  • The superseded machinery is gone: russh, russh-cryptovec and tokio from [patch]; rsa, pageant, the internal-russh forks, p384, fiat-crypto and the rest of the RustCrypto closure from Cargo.lock; p384_backend from .cargo/config.toml. git grep finds no russh, p384_backend or build/ssh outside issues/, and the issue references that remain are history.
  • The move to lib/ssh follows step 5 of the tree's placement rule, and its references are updated.
  • One P-256/P-384 path on untrusted input existed: russh's ecdsa-sha2-nistp384 verify. It is deleted, and only Ed25519, X25519 and ML-KEM remain.
  • The sshd(8) and MaxStartups/LoginGraceTime citations are documentation, not another implementation's source.

There are two problems in that model, B1 and B2. One concerns the resource bound this PR claims for unauthenticated clients; the other is a host regression in reaping.

Findings

BLOCKER B1 system/sshserver/src/connection.rs:125,151 and :84,101-110: the login deadline does not bound a client that stops reading, and the socket's input queue is unbounded.

  • The deadline is enforced only by rx.recv_timeout (:130). The same thread writes the socket with a blocking write_all that has no timeout, so it never reaches recv_timeout while a write is stuck.
  • An unauthenticated client can make that write stick. A publickey offer without a signature for a listed key is answered PK_OK and costs no attempt (lib/ssh/src/auth.rs:18,138), and a listed public key is public by nature. A client that sends a few thousand offers and never reads fills its receive window and the server's send buffer. write_all then blocks indefinitely and the Unauthenticated place is never given back.
  • Ten such clients hold UNAUTHENTICATED_CAP permanently, so nobody can log in until restart. That contradicts main.rs:40-42 ("each for at most connection::LOGIN_DEADLINE").
  • Meanwhile read_socket keeps reading whatever the client sends into an unbounded mpsc::channel, for example IGNORE packets, which are taken after the first exchange. Daemon memory then grows without bound from an unauthenticated peer.
  • Required:
    • Give socket input backpressure. For example, the reader waits until the connection thread took its chunk, as the pumps already do, or uses a bounded queue.
    • Make the deadline bound the pre-authentication writes too. For example, use a write timeout set to the time remaining, or have the deadline shut the socket down.
    • Evidence: an end-to-end step with a non-reading client that floods offers, showing the connection closed at the deadline and the place returned. A host test is better if a loopback client can be had cheaply.

BLOCKER B2 system/sshserver/src/connection.rs:175,225-228,353-366: a program ended by its channel's or connection's end is never waited for.

  • end() kills it and drops Running, and with it the only Arc<Program>.
  • The pumps' later OutputEnd finds no channel and returns at :175 before spawning the wait.
  • On Linux and macOS, std's Child does not reap on drop, so every channel closed while its program runs leaves a zombie for the daemon's lifetime. The old daemon reaped (reap/end).
  • The PR keeps sshserver building and running on every host and uses the host build as its end-to-end oracle, so this is a regression in a supported configuration. A reap after the kill fixes it, for example a waiter thread spawned in end, or an OutputEnd for a removed channel that still waits.

BLOCKER B3 system/sshserver/src/connection.rs (tests at :417-424): the driver's logic has no committed host test.

  • The only test there is the CR LF translation. Partial-send continuation, exit only after all output is out, consumed accounting and the closed-channel and connection-end cleanup are held only by the Mac end-to-end run.
  • The PR's own control e1 proves the gap. Dropping the remainder of a partial send leaves --ci host green, and only the manual step 5b catches it (controls/summary-e1.txt).
  • Required: a host test that control e1 turns red. The cheapest is probably to put the per-channel output queue (Pending/flush) behind a send function and unit-test partial acceptance and taken ordering against a fake window. Exit-after-drain and the B2 reap could be held the same way.

BLOCKER B4: outstanding by the orchestrator's decision. The guest check of one thread blocked reading a try_cloned socket while another writes it, and of piped Child stdio used across threads, is not at b6e6912. The decision to fix the closed-output case once #1024's leaf-file cfg rule lands, keeping issues/sshserver-cannot-end-a-program-that-closed-its-output.md until then, is consistent with the head as it stands.

BLOCKER B5 system/sshserver/src/program.rs:4-6 and connection.rs:177-179: both comments say the pipes' end is "which the program's own end is what does". That is ungrammatical, and false in exactly the case issues/sshserver-cannot-end-a-program-that-closed-its-output.md records: a program can close its pipes and run on, and a child it forked can hold them after it exits. State the contract: the wait starts once both output streams reach EOF, and until then end can kill. Fix main.rs:40-42 along with B1.

SUGGESTION connection.rs:347: finish's drain invariant is cheap to keep in release. A debug_assert! that is elided in the shipped build turns a future violation into silently truncated output. An assert! fails loudly.

SUGGESTION main.rs:175: std::thread::spawn panics on spawn failure, which takes down the accept loop and the daemon. thread::Builder::spawn with the connection dropped on error contains the failure in that one connection. The spawns inside a connection thread already only take down their own connection.

SUGGESTION lib/ssh/src/authorized_keys.rs: its one user is sshserver. It is defensible in toyos-ssh as an SSH format that reuses the crate's base64 and public_blob, so this is optional. The parser itself fails closed:

  • options entries and unknown leading fields refuse;
  • the blob must re-encode exactly;
  • lines over 8 KiB are refused;
  • the tests and control m1 hold the load-bearing refusal.

SUGGESTION (cosmetic): issues/fork-estate-outside-the-warning-bar.md:31, issues/the-t14-reboots-through-ubuntu-for-every-test.md:20 and lib/update/Cargo.toml:23-25 were left with unwrapped or stub lines after the edits.

Records.

  • Closing the-tokio-forks-root-manifest-patches-mio.md with its literal exit unmet is acceptable: the defect existed only for a consumer of the fork, and none remains.
  • The track's stage 2 now says the sftp replay runs against the in-memory server, which the orchestrator accepted.
  • The crypto track states the owner's 2026-10-22 ruling as the reason toyos-crypto ships before its stages 3 and 4.

Evidence.

  • --ci host was green at ef4292cd0. The head's code differs by the 7-line finish change (git diff ef4292cd0 cdd3ddff2 confirms it), and tests and clippy pass at the head. Both images build at ef4292cd0.
  • The three new recordings negotiate the hybrid and replay.
  • The end-to-end run (e2e/run.rev = cdd3ddf) shows all 17 steps as described, including the cap (the eleventh connection gets no greeting) and the 120 s close of an idle client. That idle client never had output pending, so it does not exercise B1.
  • For the orchestrator: e2e/run.log publishes host home paths (/Users/<name>/…), which is personal data in published evidence.

SEND BACK for B1–B5.

Base automatically changed from wt/ssh-mlkem to main October 22, 2026 21:55
Japabu added a commit that referenced this pull request Oct 22, 2026
…, ended programs are reaped

- A client that floods offers, each answered PK_OK at no cost, and never
  reads could block the connection's write forever and hold its
  unauthenticated place. Before login every write now has the time left
  to the deadline as its timeout, so such a connection is closed at the
  deadline and gives its place back. The socket's reader waits until its
  chunk is taken, as a pipe's reader does, so a client that sends faster
  than the connection answers waits on its socket instead of filling
  memory.
- A program ended by its channel's or connection's end is killed and
  reaped, so a host keeps no zombie.
- A channel's waiting output is a type of its own, held by a host test of
  a narrow window, which dropping the rest of a partial send turns red.
- serve takes its authorizer and deadline, so a host test drives it over
  loopback with a client that floods offers and never reads.
- finish's drain invariant is an assert. A connection thread that cannot
  be spawned closes that connection rather than the daemon.
- Comments state the contracts: the wait begins once both output streams
  reach end of file, and until then a program can be killed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Round 1 answered at 81e922b (04f7df6, 6fa3a2d, 81e922b). Evidence, all masked: https://github.com/ToyOSOrg/evidence/tree/7af56087898bea4b8953bb1b7484f51d498ebc78/pr-1026/r1

  • B1:
    • Before login, every write takes the time left to the deadline as its timeout (Connection::write_out), and so does the refusal's DISCONNECT.
    • The socket's reader hands over one chunk and waits until it is taken (Input::Socket { bytes, taken }).
    • a_client_that_floods_offers_and_never_reads_is_closed_at_the_deadline drives serve over loopback with a client that floods unsigned offers and never reads. To make that possible, serve now takes its authorizer and its deadline.
    • Control c2, an hour instead of the time left, reds it on its minute bound.
  • B2: Program::end kills, then waits. a_program_ended_is_reaped holds it, and control c3 (no wait) reds it. The Mac end-to-end run counts no zombie after step 10.
  • B3: a channel's waiting output is now Output, flushed through a send function. output_goes_whole_and_in_order_through_a_narrow_window holds it, and control c1, which is e1, reds it.
  • B4: stream_handles gains its two-thread part and its piped-children part. It ran green on QEMU x86-64 through a temporary row, since a discovered binary otherwise runs only in the T14's shared boot. With the peer never ringing, the run stalls red.
  • B5: the contracts in program.rs, the OutputEnd arm and main.rs's cap now say what holds.
  • Suggestions:
    • Taken: the release assert!, Builder::spawn, and the re-wrapped lines.
    • Kept: authorized_keys in toyos-ssh.
  • Privacy: every published log on pr-1023 and pr-1026 is re-masked. The earlier pass skipped run.log, because a non-UTF-8 byte made the search treat it as binary.
  • --ci host at 81e922b: 101 steps green. The aarch64 image builds.

@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1026, round 2 (head 81e922b; fixes in 04f7df6, 6fa3a2d and 81e922b)

Scope: round 1's B1–B5, the code the fixes added (+505/−83: about +60 net production in connection.rs, program.rs and main.rs; about +250 of host tests, including a hand-rolled flood client; +160 of guest check), and the new evidence tree.

Architecture. The fixes stay inside round 1's model.

  • Socket input now gets the same one-chunk-in-flight handshake the pumps use, so there is no second mechanism.
  • Output is the per-channel queue made testable, and it adds no layer.
  • serve now takes its authorizer and its deadline. That serves a real test and leaves the policy in main.rs.

Base. The PR already targets main. GitHub reports it CONFLICTING, and git merge-tree origin/main 81e922b27 conflicts in Cargo.lock, lib/update/Cargo.toml and issues/toyos-crypto-is-in-the-tree-on-par-with-ring-and-secure.md. These come from #1020. The rebase changes Cargo.lock, so --ci host needs to run again on the rebased head.

Round 1 blockers

B1 (pre-auth deadline, unbounded input): input resolved; the write bound is incomplete. See BLOCKER 1.

  • read_socket now waits for taken before reading again (connection.rs:139-150). Daemon memory per unauthenticated client is therefore bounded by one 64 KiB chunk plus the output it produces.
  • Every pre-auth write_out gets a timeout, and so does the refusal's DISCONNECT.
  • The loopback flood test is a sound oracle for a client that stops reading. Control c2 reds it on its minute bound, which shows the test really exercises a stuck write.

B2 (zombies): resolved. Program::end kills and then waits (program.rs:41-46), the Mac run shows no zombie after step 10, and the code establishes the fix on inspection. See SUGGESTION 1 for the test.

B3 (driver host test): resolved.

  • Output::flush is the code the connection actually uses, called through Connection::flush.
  • output_goes_whole_and_in_order_through_a_narrow_window covers continuation after a partial send, ordering across stdout and stderr, and taken only once a chunk has gone whole.
  • c1 (round 1's e1) reds it.
  • The release assert! in finish holds exit-after-drain.

B4 (guest concurrency check): resolved.

  • stream_handles has two new parts. The first is one end used on two threads, checked by asserting that the send ring holds the write before the read is answered. The second is piped children across threads, including a kill from a thread other than the one reading.
  • It ran green on QEMU x86-64 through a temporary row whose patch is in the evidence. The never-rung control stalls red, so the read really did wait while the write completed.
  • The committed registration is the T14's shared boot, as the header states.

B5 (false comments): mostly resolved. The OutputEnd arm and program.rs's first two sentences now state the contract. One clause is still false; see BLOCKER 2.

Findings

BLOCKER 1 system/sshserver/src/connection.rs:201-210, together with :17-19 and main.rs:42-43: the deadline bounds each send, not the whole write.

  • set_write_timeout sets SO_SNDTIMEO. Linux socket(7) and macOS setsockopt(2) both say a send blocked for that long "returns with a partial count". write_all then calls write again, and each new call can block for the full timeout once more.
  • So a client that reads a little before each timeout runs out keeps one write_out going for many times the remaining time. Each call returns a partial count, and the loop starts the next one.
  • This is not the "stops reading" case the test covers. But the header ("the deadline bounds everything ... every write to it") and UNAUTHENTICATED_CAP's doc ("closed at LOGIN_DEADLINE whatever it does") claim this exact bound, and it was round 1's B1 requirement.
  • Fix, a few lines: replace write_all with a loop of write calls that sets the timeout to deadline - now before each call and fails at zero.
  • Alternatively, give the deadline a thread that shuts the socket down when it fires. That bounds every blocking operation at once.
  • No new end-to-end run is needed. The flood test still holds the stuck-write case, and inspection settles the per-call arithmetic.

BLOCKER 2 system/sshserver/src/program.rs:6-7: ", or whose own child holds them, is waited for and cannot be ended" is false.

  • If a child of the program holds the pipes, the streams never reach end of file. So the wait never starts, and end's try_lock succeeds: the program is killed and reaped.
  • What actually goes wrong is different. The child survives the connection's end, its pumps stay blocked on the pipes, and the channel reports no exit while the child holds them.
  • Fix: delete the clause. If the child case is to be recorded, add one sentence to issues/sshserver-cannot-end-a-program-that-closed-its-output.md saying what really happens.

SUGGESTION 1 system/sshserver/src/program.rs:50-62: a_program_ended_is_reaped cannot tell whether end reaped the program.

  • After a wait, std caches the status. Without the wait, the test's own try_wait calls waitpid(WNOHANG) and reaps the process itself whenever SIGKILL has already landed.
  • So c3 is red only because it loses the kill-to-death race, and a faster host could turn the control green.
  • The green direction is deterministic, and two lines of inspection establish the reap. Either delete the test, as root's rule against testing what inspection establishes suggests, or keep it knowing its control is a timing race.
  • Its failure message ("the killed program was reaped") also states the success case.

SUGGESTION 2 (cosmetic): two small points.

  • serve's parameter deadline: Duration becomes login in run, which uses the name deadline for an Instant. Calling the parameter login in both places would avoid the clash.
  • In issues/the-t14-reboots-through-ubuntu-for-every-test.md:21, the rewrap moved the overlong line instead of removing it.

Evidence and privacy

  • Code changes are covered: --ci host at 81e922b (exit 0, 101 green), aarch64 --build-only exit 0, x86_64 built by the QEMU row, controls c1–c3 at 04f7df6, and the e2e at 04f7df6. No code changed after 04f7df6 except the guest check, which has its own run, and the issues/ and Cargo comment text.
  • I scanned the evidence tree at 7af5608 with binary-safe matching. It has no /Users/, home directory, user name, MAC or non-loopback address. Home paths appear only as <scratch>.
  • Remaining: pr-1026/ci-host.log, ci-host-old-base-licence-red.log and r1/ci-host.log carry the macOS per-user temp path /private/var/folders/<hash>/T/.... That directory name is specific to the user and the machine, so mask it as the home paths are. storage: toyos-blockdev, with fat32, Served and every consumer async over it #1025's evidence has the same path, so this belongs to the orchestrator's masking pass and is not a code blocker.

SEND BACK for BLOCKER 1 and BLOCKER 2.

Japabu and others added 6 commits October 23, 2026 01:02
sshserver drives toyos-ssh's sans-IO server from one thread per
connection. Every event reaches that thread on one queue: the socket's
bytes, a program's output, what its stdin took, and its exit. That
thread alone touches the protocol and writes the socket. Flow control is
the protocol's end to end. The client's window comes back only as a
program's stdin takes the bytes, so the old 30 s input stall is gone. A
pipe's reader waits until its chunk is in the client's window before
reading again.

It keeps what it served: a shell (with CR LF for a terminal), exec
through command.rs, and the sftp subsystem over sftp.rs. It adds what
russh did not give:
- several channels on one connection, keyed by toyos-ssh's ChannelId,
  which closes the input aliasing;
- an authorized_keys reader in toyos-ssh, after sshd(8)'s format, where
  an entry with options authorizes nothing;
- a login deadline of 120 s, which bounds a client that only offers keys;
- a cap of 10 connections that have not logged in, after which one more
  is closed at once.

toyos-ssh gains the requests this needs: shell, the sftp subsystem and
one pty-req before a program, and an Authenticated event that names the
key's fingerprint. The host key is shared by Arc.

A program is ended when its connection or channel ends. std's Child takes
wait and kill through one &mut, so the wait begins once both of its
pipes are at their end. A program that closed them and runs on is not
ended (issues/sshserver-cannot-end-a-program-that-closed-its-output.md).
Before, a try_wait loop with a backoff polled for it.

toyos-ssh now has two users, the build and sshserver, so it moves to
lib/ssh under the tree's placement rule.

Deleted: russh, russh-cryptovec and tokio from the root [patch], and
with them rsa (RUSTSEC-2023-0071), pageant, internal-russh-num-bigint,
internal-russh-forked-ssh-key, p384 0.14 and fiat-crypto 0.3 from
Cargo.lock; and the p384_backend setting.

Recorded with OpenSSH_10.3p1 against toyos-ssh's scripted driver, each
on the hybrid key exchange and each replayed byte for byte:
- a shell with a terminal;
- an sftp session, whose server is in memory so its replies are the same
  on every host;
- two channels open at once through one multiplexed connection.

Closes issues/sshserver-holds-one-channel-and-does-not-say-so.md,
issues/sshserver-still-speaks-rsa.md and
issues/the-tokio-forks-root-manifest-patches-mio.md (no consumer).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
…, ended programs are reaped

- A client that floods offers, each answered PK_OK at no cost, and never
  reads could block the connection's write forever and hold its
  unauthenticated place. Before login every write now has the time left
  to the deadline as its timeout, so such a connection is closed at the
  deadline and gives its place back. The socket's reader waits until its
  chunk is taken, as a pipe's reader does, so a client that sends faster
  than the connection answers waits on its socket instead of filling
  memory.
- A program ended by its channel's or connection's end is killed and
  reaped, so a host keeps no zombie.
- A channel's waiting output is a type of its own, held by a host test of
  a narrow window, which dropping the rest of a partial send turns red.
- serve takes its authorizer and deadline, so a host test drives it over
  loopback with a client that floods offers and never reads.
- finish's drain invariant is an assert. A connection thread that cannot
  be spawned closes that connection rather than the daemon.
- Comments state the contracts: the wait begins once both output streams
  reach end of file, and until then a program can be killed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
…eads

sshserver reads each connection on one thread and writes it on another,
and runs every program with its stdin, stdout, stderr and wait on
threads of their own. Two checks hold what it relies on, in the guest
because std's ToyOS arms build only for toyos:

- A client writes on one thread while its other thread is in a read the
  peer has not answered. The write lands in the send ring, and the read
  then takes the bytes the peer publishes after it.
- A child's stdin is written, its stdout and stderr are read and its exit
  is waited for, each on its own thread, with more than a pipe holds in
  flight. A child that holds its pipes open is killed from a thread other
  than the one reading them, and that read ends.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
A write timeout bounds one send. A reader that takes a little at a time
let write_all go on past the login deadline, because each send made
progress. Each send is now given only the time left (write_by), held by
a test whose reader takes one byte a send. Round 1's single timeout
over write_all reds it.

program.rs no longer claims a program whose own child holds its pipes
cannot be ended. The closed-output issue records what happens: it is
reaped when its channel or connection ends, and the child runs on.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu
Japabu force-pushed the wt/sshserver-on-toyos-ssh branch from 81e922b to 2b38b4f Compare October 22, 2026 23:28
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Round 2 answered at 2b38b4f, rebased onto main 35e9301. Evidence: https://github.com/ToyOSOrg/evidence/tree/28a0e72a9e398dafb29069d2679186e54d6f1e98/pr-1026/r2 (built on c79c17fef; <tmp>/ masked).

  • Write deadline: write_by gives each send only the time left, so the deadline bounds the whole write.
    • a_write_to_a_slow_reader_ends_at_the_deadline holds it, with a sink that takes one byte a send.
    • Round 1's single timeout over write_all reds it: 200 bytes over 5.4 s against a 200 ms deadline.
    • I first tried a loopback test that reads at 4 KiB/s. It passed under round 1's code too: the loopback buffers absorbed the answers, so no write was in progress at the deadline. I dropped it.
  • program.rs: the false clause is deleted. The closed-output issue now records what happens when a program's own child holds its pipes: it is reaped when its channel or connection ends, and the child runs on.
  • Rebase: the conflicts with build, doom: the host's HTTPS on toyos-crypto's rustls provider #1020 are resolved. lib/update takes main's comment, and the crypto table keeps main's lib/update row with this PR's SSH rows. --ci host at 2b38b4f is green, 101 steps.

@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1026, round 3 (head 2b38b4f, rebased onto main 35e9301)

Scope: round 2's two blockers and the code their fixes added in 2b38b4f (+66/−12). GitHub now reports the PR MERGEABLE against main.

Round 2 blockers

BLOCKER 1 (the deadline bounded each send, not the whole write): resolved.

  • write_by (system/sshserver/src/connection.rs:399-420) sets the timeout to the time left before every write. It fails when no time is left, on a zero-length write, and on any error other than Interrupted. That bounds the whole pre-auth write by the deadline, whatever each send achieves.
  • write_out with None sets no timeout (set_write_timeout(None)), so writes after login behave as before.
  • a_write_to_a_slow_reader_ends_at_the_deadline drives the loop directly with a sink that takes one byte every 20 ms against a 200 ms deadline. The timing is robust in both directions:
    • sleep only lengthens the run, so at most about 10 bytes go.
    • Writing all 200 bytes would take at least 4 s.
  • Control d1, round 1's single timeout followed by write_all, reds it: 200 bytes written over 5.4 s (log inspected).
  • The README records why the loopback slow-reader variant was dropped. It passed under round 1's code too, so it could not catch this defect. Dropping it was the right call.

BLOCKER 2 (the false clause in program.rs): resolved.

  • program.rs:6-7 now states only the closed-output case, which is true.
  • The issue gains one accurate sentence: a program whose child holds its pipes is not waited for while they stay open, is reaped when its channel or connection ends, and the child runs on.

Evidence

  • pr-1026/r2/ci-host.log: --ci host at 2b38b4f exits 0 with 101 steps green. It reflects the rebase's Cargo.lock change.
  • I scanned the whole evidence tree at 28a0e72 with binary-safe matching. It contains no /private/var/folders, /private/var, /Users/ or user names. The only /home/ hits are the masked <scratch>/e2e/home/..., and the logs use <home>, <tmp> and <worktree>.

No blockers remain.

LAND

@Japabu
Japabu marked this pull request as ready for review October 22, 2026 23:31
@Japabu
Japabu added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 63bc8a3 Oct 23, 2026
8 checks passed
@Japabu
Japabu deleted the wt/sshserver-on-toyos-ssh branch October 23, 2026 00:02
Japabu added a commit that referenced this pull request Oct 23, 2026
- A `login` row holds no launcher of the machine's session
  (`holds_machine_launcher`): its launches are its sessions' alone. The
  compositor started with no desktop session opens one, and again at each
  launch asked for while none is open, saying why it is refused; the
  supervisor refuses every launch into a session that has ended. A `login`
  test-runner opens one session for its jobs and hands them its launcher.
- A session's launcher acceptor lives in its `Login` alone, so its end closes
  the port and a launch connected to it reads its end. The screen's row is
  handed its session's launcher and control minted for each start. The
  compositor's launcher thread takes a copy of the reopened launcher and
  holds no lock across a launch (`Namespace::duplicate`).
- Reopens are budgeted on failures: a login authority whose sessions failed
  more than three times in ten seconds is refused its next open, and no open
  spends anything, so one session's failure or a sibling's sign-out changes no
  other login.
- The last session slot is the desktop's, and an owner that breaks its link
  is killed, its session ending as a failure.
- `session-admin` has no user dimension: the machine has one user, so a login
  session's `logout` acts on every session.
- sshserver's session half is rewritten on #1026's server: a connection opens
  its session with its first program and signs it out when it ends, and a
  channel's end asks its program to quit with a hang-up through a handle of
  its own (`spawn::Ender`), on a host still killing it.
- `app_view` loses its machine-session grants arm, since a `login`
  test-runner's jobs launch in its session.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu added a commit that referenced this pull request Oct 23, 2026
- A `login` row holds no launcher of the machine's session
  (`holds_machine_launcher`): its launches are its sessions' alone. The
  compositor started with no desktop session opens one, and again at each
  launch asked for while none is open, saying why it is refused; the
  supervisor refuses every launch into a session that has ended. A `login`
  test-runner opens one session for its jobs and hands them its launcher.
- A session's launcher acceptor lives in its `Login` alone, so its end closes
  the port and a launch connected to it reads its end. The screen's row is
  handed its session's launcher and control minted for each start. The
  compositor's launcher thread takes a copy of the reopened launcher and
  holds no lock across a launch (`Namespace::duplicate`).
- Reopens are budgeted on failures: a login authority whose sessions failed
  more than three times in ten seconds is refused its next open, and no open
  spends anything, so one session's failure or a sibling's sign-out changes no
  other login.
- The last session slot is the desktop's, and an owner that breaks its link
  is killed, its session ending as a failure.
- `session-admin` has no user dimension: the machine has one user, so a login
  session's `logout` acts on every session.
- sshserver's session half is rewritten on #1026's server: a connection opens
  its session with its first program and signs it out when it ends, and a
  channel's end asks its program to quit with a hang-up through a handle of
  its own (`spawn::Ender`), on a host still killing it.
- `app_view` loses its machine-session grants arm, since a `login`
  test-runner's jobs launch in its session.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu added a commit that referenced this pull request Oct 23, 2026
- A `login` row holds no launcher of the machine's session
  (`holds_machine_launcher`): its launches are its sessions' alone. The
  compositor started with no desktop session opens one, and again at each
  launch asked for while none is open, saying why it is refused; the
  supervisor refuses every launch into a session that has ended. A `login`
  test-runner opens one session for its jobs and hands them its launcher.
- A session's launcher acceptor lives in its `Login` alone, so its end closes
  the port and a launch connected to it reads its end. The screen's row is
  handed its session's launcher and control minted for each start. The
  compositor's launcher thread takes a copy of the reopened launcher and
  holds no lock across a launch (`Namespace::duplicate`).
- Reopens are budgeted on failures: a login authority whose sessions failed
  more than three times in ten seconds is refused its next open, and no open
  spends anything, so one session's failure or a sibling's sign-out changes no
  other login.
- The last session slot is the desktop's, and an owner that breaks its link
  is killed, its session ending as a failure.
- `session-admin` has no user dimension: the machine has one user, so a login
  session's `logout` acts on every session.
- sshserver's session half is rewritten on #1026's server: a connection opens
  its session with its first program and signs it out when it ends, and a
  channel's end asks its program to quit with a hang-up through a handle of
  its own (`spawn::Ender`), on a host still killing it.
- `app_view` loses its machine-session grants arm, since a `login`
  test-runner's jobs launch in its session.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant