Skip to content

crypto: no out-of-line calls on ChaCha20's per-message path (parity item 9, part) - #1015

Merged
Japabu merged 1 commit into
mainfrom
wt/toyos-crypto-perf-8
Oct 22, 2026
Merged

Japabu merged 1 commit into
mainfrom
wt/toyos-crypto-perf-8

Conversation

@Japabu

@Japabu Japabu commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Based on main (d7f269d); independent of #1013 and #1014. One commit, 74aa231.

Problem

The track records a per-message constant on ChaCha20-Poly1305: 17–51 ns over ring at 0 and 64 bytes, on both benches. At #1000 on the T14, seal/0 was 1.18 and seal/64 1.12; on the AMD bench (probe fb93764) seal/0 was 1.26 and seal/64 1.34.

In [profile.toyos] (x86-64), every ChaCha20 message made out-of-line calls that do no work:

  • arch::x86_64::features(), once per dispatch decision, several times per message;
  • four calls to offset_from_unsigned's debug precondition check (runtime_ptr_ge) in initial, the AVX paths' state construction, from copy_from_slice and zip over slice chunks.

What changes

  • initial builds the state by indexing.
  • features() inlines its cached atomic load; only the first read stays out of line, #[cold].
  • Result, by inspecting the shipped x86-64 code: the instruction paths reached from AEAD seal and open and from OpenSSH seal, open and packet_length call neither any more. The remaining precondition calls are in the portable AES and GCM fallbacks, which no AES-NI machine runs.

This is part of item 9 (its C2 and C4 constant). The stitched AVX2 kernel (C1) is already on main. Open's keystream stitch (C1a) and the short-message reorganisation (C2) need bench iterations.

Checks

All at 74aa231. Logs: https://github.com/ToyOSOrg/evidence/tree/b82a36fafc5dcd508a202ef44fdd09e9c1272157/pr-1015/r1

  • cargo run -- --ci host: exit 0, 93 steps green.
  • Crate clippy (adopted lints, -D warnings) in the three CI forms, for the host and --target x86_64-apple-darwin: exit 0. Crate tests in the three forms: exit 0. x86_64-unknown-uefi build: exit 0.
  • cargo test -p toyos-crypto --features std,sign,wpa2,rustls-provider --release --target x86_64-apple-darwin (Rosetta): exit 0. Rosetta reports no AVX2, so this runs SSE and AES-NI paths only, not initial.
  • crypto_instrument_prints_its_table (QEMU x86_64): exit 0. The harness's CPU model has AVX2, and before timing crypto_parity checks ours against ring at every length from 0 to 1153 B, through this AVX2 path. That is the test that reaches initial.
  • initial produces the same 16 words as before: SIGMA, the eight key words, two zero counter words, the two nonce words. Inspection settles that; no new test.

Measurements

Readback staged at 74aa231 (crypto_instrument, cryptocase; exit 2 = Staged). The rows to read are the ChaCha20-Poly1305 and OpenSSH ones at 0, 64 and 1024 B on both benches. I don't know how much of the constant these calls are; this run measures it.

Risks

None known beyond the measurement still to come. GCM's 1 KiB shipped-profile loss on the T14 (G7) was not found here: the VAES and AES-NI kernels make no out-of-line call in either profile.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q

In the shipped profile every ChaCha20 message called out of line:
- `features()` once per dispatch, several times per message;
- the debug precondition check of `offset_from_unsigned` four times in
  the AVX state's construction, from `copy_from_slice` and `zip` over
  slice chunks.
These are part of the per-message constant the track records on both
benches (17 to 51 ns at 0 and 64 bytes).

`initial` now builds the state by indexing, and `features()` inlines its
cached load, leaving only the first read out of line and cold. The
shipped x86-64 code of AEAD and OpenSSH seal, open and packet_length then
calls neither on any instruction path; the remaining precondition calls
are in the portable AES and GCM fallbacks, which no AES-NI machine runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PXEk8BBMDT8tXu4AnENB9Q
@Japabu

Japabu commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

Review of #1015, round 1 (head 74aa231, item 9 part)

I read the commit, arch/x86_64.rs's feature cache and chacha.rs's initial at the head. I ran nothing.

What was removed

Nothing safety-relevant.

  • initial.
    • The removed precondition calls are offset_from_unsigned's debug checks inside copy_from_slice and zip over as_chunks. They are library-internal UB checks of pointer arithmetic on our own fixed-size arrays, not a validation of any input.
    • The replacement indexes [u8; 32] and [u8; 8] with constant offsets: 4i + 3 ≤ 31 for the key and ≤ 7 for the nonce. The compiler checks those bounds and folds them.
    • The output is the same 16 words: SIGMA, the eight key words, the two zero counter words (12 and 13), then the two nonce words. I checked this against the old code.
    • It stays in a Secret, so erasure is unchanged.
  • features().
    • The cached load is inlined. The first read is #[cold] and #[inline(never)].
    • The portable override and the READ bit, which separates "unread" from "read with no features", are unchanged.
    • Two threads racing on the first call both store the same value, and Relaxed suffices because the word publishes no other memory.
    • FEATURES moved from function scope to module scope, and only features() and first_features() touch it.

Evidence

  • --ci host, clippy in three forms for both host targets, and the crate tests all exit 0, as does the UEFI build.
  • The body is right that Rosetta (no AVX2) does not reach initial.
  • QEMU's crypto_parity checks ours against ring at every length from 0 to 1153 B on the AVX2 path, before timing. That run is the test that executes the new initial, and it exits 0.
  • No timing claim is made until the bench run.

Findings

No blockers.

SUGGESTION

  1. x86_64.rs — the doc on features() ("Inline: every instruction path asks, several times per message.") explains the attribute, so it earns its place. Optionally fold it with the cache comment above it into one line.

PR body (orchestrator): the base is d7f269d, not 81c1386.

Verdict: LAND.

@Japabu
Japabu marked this pull request as ready for review October 22, 2026 13:13
@Japabu
Japabu added this pull request to the merge queue Oct 22, 2026
Merged via the queue into main with commit 18a3873 Oct 22, 2026
8 checks passed
@Japabu
Japabu deleted the wt/toyos-crypto-perf-8 branch October 22, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant