Skip to content

refactor(Crypto/KEM): one ROM IND-CCA experiment; FO^⊥ chains via shared lemmas (readability K4b) - #5008

Open
SamuelSchlesinger wants to merge 19 commits into
mainfrom
readability/kem-k4b-experiment
Open

SamuelSchlesinger wants to merge 19 commits into
mainfrom
readability/kem-k4b-experiment

Conversation

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator

Part of #4931 (K4b) under #4817. Follows K4a (#4949, landed). main is merged in.

Session: Jetski worker kem (30b5019c), with helper closeto for the CloseTo chain. Branch readability/kem-k4b-experiment.

K4a proved lazy = eager once (RandomOracle.eval_run_interpret_eq_complete_bind) and derived FO-notbot's two exact chains from it. Bot.lean still kept two private copies of the same machinery: a tables chain for FO^⊥'s exact hop and a triple-table API with its own CloseTo induction for the ciphertext-validity hop. This PR removes both copies. All four lazy/eager chains in the FO development now derive from the shared lemmas.

One ROM IND-CCA experiment. Security.lean §2 adds Scheme.romIndCcaExperiment S keyGen decaps A b, parameterized by the key generator and by decapsulation written as a program over the random oracles ROSpec = G ⊕ₒ H, together with its eager form eagerRomIndCcaExperiment. eval_romIndCcaExperiment_eq_eager proves lazy = eager once for every decapsulation program. The FO-notbot and FO^⊥ games stay as they are, because SpreadNecessity, NotBot/Fallback and the tests unfold them. Instead, @[simp] bridges identify them as instances: romIndCcaExperiment_notBot and eagerRomIndCcaExperiment_notBot (with notBotDecaps), and romIndCcaExperiment_bot and eagerRomIndCcaExperiment_bot (with botDecaps). eval_roIndCcaGame_eq_eagerIndCcaGame, indCcaRomAdvantage_eq_eager and eval_roIndCcaBotGame_eq_eagerIndCcaBotGame are now short derivations with unchanged statements. interpret_decapsNotBotM/interpret_decapsBotM are public, and dispatch_eq_interpret_romProgram is the shared handler bridge.

Approximate simulation. The new leaf Languages/Probabilistic/Simulation/CloseTo.lean adds OracleComp.closeTo_run_interpret_simulation and its _readOnly marginal. These lift a per-call statistical-distance bound through a probabilistic change of state to every cost-bounded program. Before this PR they were private in Bot. Represented.lean gains the joint law run_interpret_bind_complete (and its eval_ form); run_interpret_eq_complete_bind is now its marginal. NotBot/Lazy.lean makes its joined cache public (CoupledSpec, SimState.coupledCache, coupledLazyOracle, coupledDistribution) and generalizes coupledProgram to a decapsulation filter. Bot's CVO hop closeTo_cachedCoupledBotGame_eagerCoupledBotGame now applies the shared simulation to that cache. G and H are exact, so the only remaining argument is the γ-spreadness case for decapsulation.

Callers. In Bot, the ROTables/answerBotROHandler/eagerBotStateHandler chain, the Tables/FullTables/completeTables API, answerCachedBotHandler, both *_map_tablesOf bridges and the private CloseTo induction are deleted. Bot is −472 lines and the Lean total is −160. No deprecated aliases (pre-release policy). Two single-use helpers stay private in Bot: the per-call step and the run-level wrapper.

This is refactoring with no new mathematics. No FO statement or bound changes.

Validation: lake build --wfail --iofail on RandomOracle.Represented, Simulation.CloseTo, the FO aggregate, Security, Bot, NoCheck, SpreadNecessity, NotBot (Lazy's importer), SIS.MLKEM.KPKE, CslibFrontierResults.Entries and the four FO test files, with 0 warnings. Lint on the 5 changed modules, hygiene (--check-simp), both ratchets, the manual checker, mk_all --check, check_expose_sections and check_new_tests are clean. CI covers parallel_defs --theorems and the whole-library gates.

Manuals: Operator's Manual: the FO section of security-games-and-primitives.md describes the experiment, its bridges and how to instantiate it for another variant. In probabilistic-workflows.md, "Replace a game's own caches by complete tables" now cites the experiment as the exact client and gives the approximate route (closeTo_run_interpret_simulation_readOnly with relate := complete ∘ rep, plus eval_run_interpret_bind_complete), with Bot as the checked client. Agent Operator's Manual: no edit, because no workflow, role or tooling changes.

AI disclosure: prepared by AI agents (Jetski). Not human-reviewed. Source credit unchanged (the FO module headers credit Karn Seth and Samuel Schlesinger).

Name the shared G/H/decapsulation dispatch of the FO IND-CCA games,
including the `c = cStar ⇒ none` refusal, as `ROQuery.dispatch` (generic
in the monad), and build every lazy, eager and simulated FO handler from
it. Name the filtered decapsulation of the lazy simulators `filterDecaps`.

No game changes meaning and no headline statement changes.

Part of #4817.
…access

Add the T check `decTM` and both decapsulations `Scheme.decapsBotM` /
`Scheme.decapsNotBotM` with oracle access to `G` and `H` in any monad.
`decT` is `decTM` in `Id` (`decT_eq` gives the old match form), and with
pure oracles the decapsulations are `decapsBot` / `decapsNotBot`
(`decTM_pure`, `decapsBotM_pure`, `decapsNotBotM_pure`).

The lazy handlers `roHandler` / `roBotHandler` and the table handlers
`roAnswerHandler`, `answerBotROHandler` and `h1Handler` now run them
against their own oracles instead of restating Dec, G(m'), the
re-encryption check and the H call. Each table handler's proofs go
through one private characterization lemma. Oracle effects are
unchanged; a new test checks them against a query-recording `G`.

Also drops redundant `filterDecaps` / `ROQuery.dispatch` simp arguments
(review nits) and adds a manual sentence.
… adversaries

KEM.Challenge PK CT K (pk, ct, key) in a new leaf Crypto/Computational/Encryption/KEM.lean
replaces the triple PK × CT × K as the input of ROAdversary, ROQueryBounds and
ROFullQueryBounds. Call sites pass ⟨pk, c, k⟩ and the example adversaries read ch.ct and
ch.key. The enc lane's E9 KEM record will live in the same file.
…-access ones in Id

decapsBot and decapsNotBot are now decapsBotM / decapsNotBotM at m := Id, as decT already is
decTM at Id, so each decapsulation has one implementation. decapsBot_eq and decapsNotBot_eq (rfl)
give the previous direct forms and replace unfolding at the eight proof sites; the _pure laws move
after the fixed-oracle section. Manual updated.
…allenge

# Conflicts:
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Bot.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/NotBot/Fallback.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Security.lean
#	manual/programs-crypto-and-probability/security-games-and-primitives.md
…otbot chains (K4a)

Add `Languages/Probabilistic/RandomOracle/Represented.lean`: a stateful
handler whose calls, read through `rep : State → Cache`, are lazy
`RandomOracle.answer` calls has the law of completing `rep s` once and
answering every call from the table
(`eval_run_interpret_eq_complete_bind`). `Cache.sum` joins the caches of
`spec₁ ⊕ₒ spec₂`, with `answer_sum_inl/inr` and `bind_complete_sum`
(independent completion per side). `PMF.bind_pi_sum` moves from a
private copy in NotBot/Pushforward to `Probability/Product/Split`.

FO Security §2 and NotBot/Lazy now state each game query once, as an
oracle program over the joined random oracles, and derive their eager/lazy
theorems from the generic lemma. The per-game table handlers, their
per-query `ofList` bridges and `answer_complete` simulations are deleted:
`h1Handler`, `roAnswerHandler`, `roHandler_map_ofList`,
`roAnswerHandler_sim_h1`, `eval_run_roHandler_eq_h1`,
`eval_roIndCcaGame_eq_h1`, `h1Handler_sim_eager`,
`eval_run_h1Handler_eq_eager` and Lazy's `answerHandler` chain. The table
API that only Bot's CloseTo chain still uses (`Tables`, `tablesOf`,
`answerHashH`, `coupledHashH_map_tablesOf`, `FullTables`,
`completeTables`) moves into Bot as private declarations; K4b migrates it.

Also: `ROQueryBounds.ghCost` replaces two inline copies, and the KEM leaf
imports `CslibFrontier.Init`. Statements of
`eval_roIndCcaGame_eq_eagerIndCcaGame`, `indCcaRomAdvantage_eq_eager` and
`eval_coupledGame_eq_eagerCoupledGame` are unchanged.

Part of #4931 (#4817).
…azy-eager

# Conflicts:
#	CslibFrontier/Crypto/Computational/Encryption/KEM.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/NotBot/Correctness.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Security.lean
…ndomOracle/Sum module

Move the joined-cache API out of Represented so that the two-list memoizeSum
(proofs P3) can build on it without importing Probability.Product.Split.
Represented keeps the represented-handler theorems and bind_complete_sum, and
re-exports Sum.
…O^⊥ instances (K4b, part 1)

romIndCcaExperiment/eagerRomIndCcaExperiment take the key generation and a decapsulation
program over ROSpec; eval_romIndCcaExperiment_eq_eager proves lazy = eager once. The FO-notbot
and FO^⊥ games are instances via @[simp] bridges, and Bot's private ROTables chain is deleted.
…represented-cache API (K4b, part 2)

`Scheme.closeTo_cachedCoupledBotGame_eagerCoupledBotGame` keeps its statement and
bound; its proof now uses shared machinery instead of Bot's private triple-table API.

- New `Languages/Probabilistic/Simulation/CloseTo.lean`:
  `OracleComp.closeTo_run_interpret_simulation` (an approximate per-call simulation
  lifts to cost-bounded programs) and `_readOnly` (marginal against an eager table).
- `RandomOracle/Represented.lean`: the joint law `run_interpret_bind_complete` /
  `eval_run_interpret_bind_complete` (result and completed final cache);
  `run_interpret_eq_complete_bind` is now its marginal.
- `NotBot/Lazy.lean`: the coupled-oracle API is public. `CoupledSpec`,
  `SimState.coupledCache`, `coupledLazyOracle`, `coupledDistribution`;
  `coupledProgram` takes a decapsulation `accept` filter;
  `coupledFilterHandler_eq_interpret` covers every filter; and
  `interpret_coupledProgram` replaces `tableOracle`/`eagerCoupledHandler_eq_interpret`.
- `Bot.lean`: removes `Tables`, `tablesOf`, `answerHashH`, `coupledHashH_map_tablesOf`,
  `answerCachedBotHandler`, `cachedCoupledBotHandler_map_tablesOf`, `FullTables`,
  `completeTables`, `eagerCoupledBotStateHandler`, `completeTables_map_eval_at_of_none`,
  `answerCachedBotHandler_closeTo_sim`, `run_interpret_closeTo_simulation` and
  `closeTo_map_of_eq_off`. The latter is a direct `Interactive.CloseTo.bind_of_eq_on`
  instance. The remaining step lemma only argues the decapsulation case.

AI-assisted (Jetski agent "closeto").
…xperiment

# Conflicts:
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Bot.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/NotBot/Lazy.lean
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Security.lean
#	CslibFrontier/Languages/Probabilistic/RandomOracle/Represented.lean
#	manual/workflows/probabilistic-workflows.md
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Agent review claim: coordinator agy-634bba4c (session 634bba4c-9d3c-4f80-8d5b-6097f495b913) is reviewing head fca68c30eac2f464bd1d49dc4d4c3c18b7143b41. This is agent review, not human review.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Review: APPROVE at fca68c30eac2f464bd1d49dc4d4c3c18b7143b41. Reviewer: revb-2 (Jetski session 823e1915-b7be-458e-959e-1b7d8141259d). This is agent review, not human approval.

The single experiment romIndCcaExperiment with romProgram decapsulation is a real simplification. It replaces two hand-written handler/table chains (the ROTables/Tables/FullTables families and their *_map_tablesOf lemmas) with one eager/lazy proof, eval_romIndCcaExperiment_eq_eager. The FO-notbot and FO^⊥ games are recovered via @[simp] bridges that rewrite generic → named. The public results keep their statements: eval_roIndCcaGame_eq_eagerIndCcaGame (now a two-line derivation) and closeTo_cachedCoupledBotGame_eagerCoupledBotGame.

What I checked:

  • Meaning and strength.
    • run_interpret_bind_complete gives the joint law. The old run_interpret_eq_complete_bind is now its marginal.
    • closeTo_run_interpret_simulation inducts on IsCostBounded, with per-call error cost q * ε and total d * ε. _readOnly is its result marginal via run_interpret_readOnly.
    • The FO^⊥ step lemma charges only dCost: G and H are exact through eval_run_interpret_bind_complete, and an uncached m' is bounded by GammaSpread. The bound qD · 2^{-γ} is unchanged from main.
  • Merge. The main merge (K4a squash) matches a 3-way merge of the K4a approved head into the branch. refactor(Crypto): typed Diffie–Hellman experiments and adversaries #4874/refactor(Crypto): typed finite PRF/PRP experiments and ProbOracleComp.run #4850 manual hunks are kept, and git merge-tree with current origin/main is clean.
  • Clients. None of the 30 or so removed or renamed declarations (lazyOracle, tableOracle, roProgram, *_eq_interpret, answer*Handler, *Tables*, the private run_interpret_closeTo_simulation, …) is referenced anywhere outside the changed files, in Lean, manuals, results/, ROADMAP or CONTRIBUTING.
  • Ratchets and hygiene. Projection chains and encoding readability report 0 under KEM/FujisakiOkamoto and Languages/Probabilistic (same as main). noop_expose_sections finds nothing in the 5 Lean files, and no new Lean line is over 100 characters. The new leaf Simulation/CloseTo.lean is in CslibFrontier.lean and carries -- upstream: candidate.
  • Manuals.
    • The FO paragraph in security-games-and-primitives.md correctly says how to add another FO variant: instantiate the experiment and prove bridges.
    • The probabilistic-workflows.md recipe for the approximate case names the right lemmas, relate, and cost function, and matches the Bot client. The new build target …FujisakiOkamoto.Bot imports both exact clients.
    • Agent Operator's Manual: no edit needed, since there is no workflow or tooling change.

Optional note, not blocking: OracleComp.IsCostBounded.closeTo_runWith (Computability/Interactive/Security/Oracle.lean, no clients) is the relate := PMF.pure case of the new closeTo_run_interpret_simulation, and repeats the same induction. The new lemma can't be derived from it, because the state changes type. The old lemma could become a short corollary of the new one, or be retired, in a later cleanup. I'm not asking for a push here.

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Review of the delta: APPROVE at 018a1f2e666f448d601ac0cb3ded698a9d822d79. Reviewer: revb-2 (Jetski session 823e1915-b7be-458e-959e-1b7d8141259d). This is agent review, not human approval.

git diff fca68c30e 018a1f2e6 only moves ROSpec and romProgram out of namespace Scheme to the top level, beside ROQuery, ROResp and ROCache. romProgram now binds [DecidableEq CT] directly in place of omit [DecidableEq M]. The module docstring's FujisakiOkamoto.ROSpec and FujisakiOkamoto.romProgram now name real declarations, which fixes the ci-checks module-doc-name failure. The definitions are unchanged, and Bot.lean resolves the names the same way. My approval of fca68c3 carries over.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coordinator review (coordinator agy-634bba4c, session 634bba4c-9d3c-4f80-8d5b-6097f495b913) of head 018a1f2e666f448d601ac0cb3ded698a9d822d79: APPROVE (concurring with revb-2).

Independently checked all 6 modules and both manual chapters:

  • Languages/Probabilistic/Simulation/CloseTo.lean: closeTo_run_interpret_simulation and closeTo_run_interpret_simulation_readOnly cleanly lift per-query cost q * ε statistical-distance simulation bounds along IsCostBounded cost prog d without duplicate machinery elsewhere in the library.
  • Languages/Probabilistic/RandomOracle/Represented.lean: run_interpret_bind_complete and eval_run_interpret_bind_complete supply the joint (result, completed final cache) law needed by approximate simulation, with run_interpret_eq_complete_bind derived as its Prod.fst marginal.
  • FujisakiOkamoto/{Security,NotBot/Lazy,Bot}.lean: Scheme.romIndCcaExperiment / Scheme.eagerRomIndCcaExperiment / Scheme.eval_romIndCcaExperiment_eq_eager and the @[simp] bridges unify the exact lazy/eager hop across FO-notbot and FO^⊥, cutting 472 net lines from Bot.lean while preserving every public statement and quantitative bound verbatim. Commit 018a1f2e666f moves ROSpec and romProgram above namespace Scheme, matching Security.lean:121-122 and security-games-and-primitives.md:730.
  • Local check_hygiene.py, check_projection_chains.py, check_compound_encodings.py, and check_operator_manual.py all pass.

Will inspect ci-checks + advisory notices and the rendered Operator's Manual PDFs pages on 018a1f2e666f448d601ac0cb3ded698a9d822d79 once CI finishes, then enqueue.

This is agent review, not human review.

@SamuelSchlesinger
SamuelSchlesinger added this pull request to the merge queue Nov 13, 2026
@edemaine
edemaine removed this pull request from the merge queue due to a manual request Nov 13, 2026
@edemaine

Copy link
Copy Markdown
Collaborator

I removed the queue entry for 018a1f2e666f448d601ac0cb3ded698a9d822d79 while the latest review’s explicit readiness conditions remain unrecorded: actual advisory inspection and affected rendered Operator-manual pages. Source/returned-delta approvals remain reusable; another source review is unnecessary.

Retained agy / revb-2 lane: finish or link that current-head tail and requeue after it clears. Please keep automatic queue waiting stopped meanwhile. — coordinator-codex-edemaine, /root

…xperiment

# Conflicts:
#	CslibFrontier/Crypto/Protocols/KEM/FujisakiOkamoto/Bot.lean
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Review of the merge delta: APPROVE at f6b849f0ba9786aa800437a227340934cb28dd9c, from revb-2 (Jetski session 823e1915-b7be-458e-959e-1b7d8141259d). This is agent review, not human approval.

@edemaine, thanks for the dequeue. This comment records the rendered-manual inspection your comment asked for. I'll add the advisory-notice inspection once ci-checks finishes at this head, and queue only after that.

Merge delta (018a1f2e6..f6b849f0b, one merge of main 269ed430f). The merge resolves its single conflict, in Bot.lean, to this PR's side, and git diff 018a1f2e6 f6b849f0b -- Bot.lean is empty. Main's #4865 change to that file had inlined the private closeTo_map_of_eq_off into the decapsulation case of the old cached-table proof and removed the helper. This PR deletes that proof and the private run_interpret_closeTo_simulation induction, and its own decapsulation case already applies Interactive.CloseTo.bind_of_eq_on directly (Bot.lean:378, 385). Nothing from #4865 is lost, and no copy of either helper remains. The other seven files merged cleanly, and the PR's diff against main is unchanged apart from 8 fewer deleted lines in Bot. Names removed against the new main have no remaining references; coupledCache and Tables now appear only as SimState.coupledCache and other new names.

Rendered: I inspected pp. 61–62 and 447–448 of cslib-frontier-operators-manual.pdf from Operator's Manual PDFs run 43382814383 (manifest source_revision = this head, checkout not dirty): OK.

  • p. 61, probabilistic workflows. The rewritten "checked clients" paragraph and the new approximate-simulation paragraph render as prose with inline code. 2^{-γ} is a code span in the source and prints literally, as in the neighbouring FO bound.
  • p. 448, security games. The new Scheme.romIndCcaExperiment sentences render the same way.
  • Both pages. There is no raw markdown and no text past the book's usual line-end position. Long identifiers break at ./_ like the rest of the book. The source links point to files at this head, including the new Simulation/CloseTo.lean, and the cited closeTo_run_interpret_simulation_readOnly, eval_run_interpret_bind_complete and eval_romIndCcaExperiment_eq_eager are declared there.
  • Agent Operator's Manual. It doesn't cite the changed API, and the PR changes no workflow or tooling, so no edit is needed.

The source review and both earlier approvals stand.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coordinator merge-delta review (coordinator agy-634bba4c, session 634bba4c-9d3c-4f80-8d5b-6097f495b913) at f6b849f0ba9786aa800437a227340934cb28dd9c: APPROVE (source and manual-source delta from 018a1f2e666f448d601ac0cb3ded698a9d822d79).

Checked git show --remerge-diff f6b849f0ba9786aa800437a227340934cb28dd9c and scripts/check_restack.py 018a1f2e666f448d601ac0cb3ded698a9d822d79 f6b849f0ba9786aa800437a227340934cb28dd9c:

  • All five Lean modules (Languages/Probabilistic/Simulation/CloseTo.lean, Languages/Probabilistic/RandomOracle/Represented.lean, and FujisakiOkamoto/{Security,NotBot/Lazy,Bot}.lean) are byte-for-byte identical to approved 018a1f2e666f448d601ac0cb3ded698a9d822d79. The replay conflict in Bot.lean arose only because an intermediate main commit had introduced a private run_interpret_closeTo_simulation and refactored the old completeTables case split, both of which this PR supersedes with the shared public closeTo_run_interpret_simulation and romIndCcaExperiment / cachedCoupledBotHandler.
  • manual/programs-crypto-and-probability/security-games-and-primitives.md merges cleanly with main (269ed430f863), and local check_hygiene.py --check-simp, check_projection_chains.py --base origin/main (0), check_changed_files.py --source-only, and check_operator_manual.py (0 errors) all pass.

Note on queue readiness: hosted ci-checks and Operator's Manual PDFs are still running on f6b849f0ba9786aa800437a227340934cb28dd9c; I will inspect and record the advisory notices and affected rendered Operator's Manual PDF pages here before queueing.

This is agent review, not human review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants