Skip to content

toyos-ssh: mlkem768x25519-sha256 first, the post-quantum hybrid of RFC 10042 (SSH stage 3) - #1023

Merged
Japabu merged 2 commits into
mainfrom
wt/ssh-mlkem
Oct 22, 2026
Merged

Japabu merged 2 commits into
mainfrom
wt/ssh-mlkem

Conversation

@Japabu

@Japabu Japabu commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

toyos-ssh offered only curve25519-sha256. The shipped server, on russh, negotiates mlkem768x25519-sha256 with OpenSSH 9.9 and later. So switching sshserver to toyos-ssh (stage 2 of issues/the-ssh-server-is-toyos-own-and-russh-is-gone.md) would have downgraded every login to a classical exchange. The owner ruled on 2026-10-22: "Use our own crypto where possible; we optimize and improve until done, but we can already use it."

Result

  • The server offers mlkem768x25519-sha256 first, then curve25519-sha256. The chosen method is the first one in the client's list that the server runs (RFC 4253 §7.1), so OpenSSH 10.3 negotiates the hybrid.
  • The method is RFC 10042 §2, now published (https://www.rfc-editor.org/rfc/rfc10042.txt):
    • C_INIT is an ML-KEM-768 encapsulation key followed by an X25519 key, 1184 + 32 bytes.
    • S_REPLY is the ciphertext followed by the server's X25519 key.
    • K = SHA-256(K_PQ || K_CL). It is encoded as a string, not an mpint, both in H and in the key derivation.
    • The primitives are toyos-crypto's ML-KEM-768 and hybrid::for_ssh.
  • Refusal::KeyExchange replaces KeyAgreement. It names its reason: a message 30 of the wrong length, an encapsulation key that fails FIPS 203 §7.2's modulus check, or an X25519 key that makes the zero secret. It disconnects with KEY_EXCHANGE_FAILED, as RFC 10042 §2.1 requires.
  • A wrong-length Q_C on the classical method now takes that reason as well, as RFC 8731 §3 recommends. Before, it disconnected with PROTOCOL_ERROR.
  • issues/toyos-ssh-offers-only-a-classical-key-exchange.md is closed. The SSH track records stage 3 as landed, and the crypto track's consumer line is updated.

Net production change: +75/−31 in build/ssh/src. Tests: +178/−49, plus the re-recorded fixtures.

Decisions

  • curve25519-sha256 stays offered, second. It serves clients older than OpenSSH 9.9. One recording forces it, so the classical path stays under the OpenSSH oracle.
  • Every recording was made again. The server's KEXINIT changed, so the old bytes cannot replay. The new recordings used the stage 1 keys, seeds and commands with OpenSSH_10.3p1. The seventh session is the forced classical one.
  • The test client runs either method. Client::handshake now offers the hybrid first, so every refusal and fuzz session opens on it. The replay test derives each method's K and keys with its own code, then opens what OpenSSH sealed.

Checks and evidence

Evidence: https://github.com/ToyOSOrg/evidence/tree/9bd93866959a422fd4854d2d1926e927798c8e84/pr-1023

Head aa65c1f, rebased onto main c8e6e03.

  • cargo run -- --ci host at c4e95d5, on the old base 18a3873: exit 0 (ci-host.log).
    • The first run at that revision failed when the development Mac's disk filled from outside this worktree (ci-host-enospc.log). The rerun had free space.
    • The later commit only puts this PR's number in the track issue.
  • After the rebase: cargo test -p toyos-ssh at aa65c1f, exit 0 (toyos-ssh-tests-rebased.log), so every recording still replays byte for byte. The rebase brought in sha2: one SHA-2, toyos-crypto's; toyos-sha2 and toyos-sha2-hw gone #1016, which makes SHA-2 toyos-crypto's; toyos-ssh reaches nothing else that changed. cargo run -- --ci host at aa65c1f: exit 0, 101 steps green (ci-host-rebased.log).
  • OpenSSH oracle. OpenSSH_10.3p1 (/usr/bin/ssh on the Mac) was the black-box client for all seven recordings.
    • Six negotiated mlkem768x25519-sha256, and none of the seven client logs has the post-quantum warning.
    • OpenSSH verified the pinned host key and the signature over H in every session.
    • Exit statuses, the echoed seq outputs and the refusals are in recordings/.
    • Each recording replays byte for byte, and again with its input cut at every byte and at every 61st.
  • NIST's ML-KEM-768 encapsulation known answers are toyos-crypto's ACVP test in lib/crypto/tests/nist.rs, which --ci host runs. RFC 10042 publishes no test vectors of its own.
  • Negative controls (controls/): five checked patches, each applied, built, tested and reverted in one run.
    • Encoding K as an mpint fails the replays and handshakes.
    • Swapping S_REPLY's halves fails them too.
    • Dropping the hybrid fails the replays.
    • Weakening the modulus check fails the hybrid-init test, and so does accepting a long C_INIT.
    • Under the first patch, the independent derivation test kept passing, as it should. It derives K with its own code and opens what OpenSSH sealed, without the server.
  • No guest or metal run: nothing built for a guest depends on toyos-ssh until stage 2.

Risks

  • K sits in a Vec and is freed without being wiped, for either method. issues/toyos-ssh-leaves-its-secrets-in-freed-memory.md records this, updated for the hybrid.
  • The fuzz's grammar still starts each re-exchange with curve25519-sha256. Drawing the method changed the fuzz's seeded sequence, and the new sequence missed one refusal kind. The OpenSSH rekey recording re-exchanges on the hybrid three times.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q

Japabu added a commit that referenced this pull request Oct 22, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Japabu and others added 2 commits October 22, 2026 22:28
…C 10042

The server offers mlkem768x25519-sha256 before curve25519-sha256, so the
switch of sshserver keeps the post-quantum exchange russh negotiates
today. The method follows RFC 10042 section 2: C_INIT is an ML-KEM-768
encapsulation key followed by an X25519 key, S_REPLY the ciphertext
followed by the server's X25519 key, and K = SHA-256(K_PQ || K_CL),
encoded as a string in H and in the key derivation. The primitives are
toyos-crypto's ML-KEM-768 and hybrid::for_ssh.

A client's message 30 of the wrong length, an encapsulation key that
fails FIPS 203 section 7.2's check, and an X25519 key that makes the zero
secret are refused as Refusal::KeyExchange, which disconnects with
KEY_EXCHANGE_FAILED as RFC 10042 section 2.1 requires. The classical
method's wrong-length Q_C now takes that reason too, as RFC 8731 section
3 recommends, rather than PROTOCOL_ERROR.

The server's KEXINIT changed, so every recording was made again with
OpenSSH_10.3p1, with the stage 1 keys and seeds. Six negotiated the
hybrid, and their client logs carry no post-quantum warning. A seventh,
forced onto curve25519-sha256, keeps the classical method under the
oracle. The test client runs either method, and the replay derives each
method's K and keys itself and opens what OpenSSH sealed.

Closes issues/toyos-ssh-offers-only-a-classical-key-exchange.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review, round 1: #1023 at aa65c1f (mlkem768x25519-sha256 in toyos-ssh)

Architecture. This is a good fit. The server's half of RFC 10042 is one more Method arm and one more METHODS entry. It sits on toyos-crypto's mlkem::EncapsulationKey and hybrid::for_ssh, and no new dependency comes in. The hybrid's validation lives at the boundary that receives C_INIT (transport.rs hybrid) and then becomes typed values: EncapsulationKey, [u8; 32] and SshSecret. Keeping curve25519-sha256 second is ordinary capability support for older clients, and a recording keeps it under the oracle. Production: +75/−31 in build/ssh/src. Tests: +178/−49, plus the re-recorded fixtures.

Protocol check against RFC 10042 §2, RFC 8731 §3 and RFC 4253 §7.2.

  • C_INIT is split at 1184 bytes, and the rest must be exactly 32 bytes. The ML-KEM key goes through FIPS 203 §7.2's modulus check (mlkem.rs from_bytes: every 12-bit coefficient below q).
  • The X25519 secret is checked for zero before the hybrid is combined.
  • S_REPLY = ct || Q_S.
  • K = SHA-256(K_PQ || K_CL) uses the raw X25519 secret, not an mpint. It is put_string-encoded and fed identically to Exchange::hash and kex::derive.
  • C_INIT and S_REPLY enter H as strings in RFC 8731's field order.
  • Every failure of message 30 maps to KEY_EXCHANGE_FAILED. A wrong-length classical Q_C now does too, which matches RFC 8731 §3.
  • A re-exchange renegotiates the method on every KEXINIT, so the rekey recording's three hybrid re-exchanges exercise the same path.
  • The test's q-coefficient mutation (bad[0]=0x01, low nibble of bad[1]=0xd) does encode 3329.

Evidence.

  • Seven OpenSSH_10.3p1 recordings. I checked the client logs: six say kex: algorithm: mlkem768x25519-sha256, curve25519 says curve25519-sha256, and none has the post-quantum warning.
  • The replay re-derives K and the keys with the test client's own combiner (hybrid_k, not hybrid::for_ssh), verifies the recorded host signature over H and opens OpenSSH's sealed packets. That makes OpenSSH the independent oracle.
  • The five controls each go red on the claim they target. m4 (>= q → > q) and m5 (a long C_INIT) are caught by the focused vector test. m1 shows the independent derivation does not depend on the server code.
  • --ci host was green at c4e95d5. The rebase diff (range-diff) touches only issue text, and cargo test -p toyos-ssh passes at aa65c1f (toyos-ssh-tests-rebased.log, exit 0). The reuse is sound.

Findings

No blockers.

SUGGESTION build/ssh/src/transport.rs:31: MLKEM768_KEY_LEN = 1184 is a private copy of a fact toyos-crypto already owns (EncapsulationKey::as_bytes() -> &[u8; 1184], and the 1184 hard-coded in from_bytes). An mlkem::ENCAPSULATION_KEY_LEN (and ciphertext length) exported by the crate would give it one home. The replay test's 1184 + 32, 1088 + 32 could use it too.

SUGGESTION build/ssh/Cargo.toml:13-16 and build/ssh/tests/replay.rs:4-11: the rewrapped comments leave stub lines ("# caller's." and "driver sent. Its own bytes —"). Reflow them.

For the orchestrator (PR description): the body links evidence tree 24ec798c…. The handoff and the current record are 58a17bf1…, which adds only the "#PR2 is #1026" note to the README. The newer link is the right one.

LAND

@Japabu
Japabu marked this pull request as ready for review October 22, 2026 21:25
@Japabu
Japabu enabled auto-merge October 22, 2026 21:25
@Japabu
Japabu added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit b43d734 Oct 22, 2026
8 checks passed
@Japabu
Japabu deleted the wt/ssh-mlkem branch October 22, 2026 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant