Repository navigation
feat: transport of means and variances under a KL constraint [MDJKLV21, Lemmas 10–12] - #291
Conversation
…1, Lemmas 10–12] Co-Authored-By: Claude Code <noreply@github.com>
AI review — approvedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
… facts - `Real.exists_pos_le_mul_sqrt_add_sub_mul_exp_sub_one_sub`: the closed-form inversion of the sub-gamma bound, shared by the KL transport and the high-probability Bernstein inequality. - `MeasureTheory.integral_mem_Icc_of_ae_mem_Icc`, `sq_sub_mem_Icc_of_mem_Icc` and `LMLExplore.absolutelyContinuous_of_klDiv_le_ofReal`, each used where the fact was inlined. - Credit the paper project Essakine2026Tight in both module docstrings. Co-Authored-By: Claude Code <noreply@github.com>
…transport-essakine2026tight-reinforcementlearning # Conflicts: # LMLExplore/Analysis/SpecialFunctions/Exp.lean
… hypothesis Co-Authored-By: Claude Code <noreply@github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0196ftMV379vy48QfzHBVNvo
There was a problem hiding this comment.
🟡 reuse — request_changes claude/claude-opus-5-5
The new transport and variance results are new and reuse Mathlib and LMLExplore API, including a shared Chernoff inversion for Bernstein. One public declaration is still a thin wrapper around Mathlib's klDiv_ne_top_iff.
LMLExplore/InformationTheory/KullbackLeibler/Bernstein.lean:79—absolutelyContinuous_of_klDiv_ne_topis a public wrapper whose whole proof is(klDiv_ne_top_iff.1 h).1. It is used at the threehμνsites (lines 158, 190, 216), and the module docstring (line 45) lists it as a main statement. Fix: Delete it and its docstring entry. At each site write(klDiv_ne_top_iff.1 (ne_top_of_le_ne_top ENNReal.ofReal_ne_top h)).1.
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 20s · 113k in (75.4k cache read, 37.6k cache write) / 1.9k out tokens · reviewing this diff · rubric ·
There was a problem hiding this comment.
Fixed in 3311c92. I deleted LMLExplore.absolutelyContinuous_of_klDiv_ne_top and removed its entry from the module docstring. The three hμν sites (abs_integral_sub_integral_le_of_klDiv_le and both variance transports) now use (klDiv_ne_top_iff.1 (ne_top_of_le_ne_top ENNReal.ofReal_ne_top h)).1, as suggested. The earlier generality finding, which turned the ≤ ofReal a form into the ≠ ∞ form, no longer applies because the declaration is gone. This head also merges current main (8280ba8), which fixes the merge conflict.
AI review — changes requestedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
|
Do check what the conflict actually is before resolving it mechanically. If |
…transport-essakine2026tight-reinforcementlearning # Conflicts: # LMLExplore/Analysis/SpecialFunctions/Exp.lean # LMLExplore/Probability/Moments/Bernstein.lean
…transport-essakine2026tight-reinforcementlearning
Remove the public `LMLExplore.absolutelyContinuous_of_klDiv_ne_top`; its three uses now call Mathlib's `klDiv_ne_top_iff` directly. Co-Authored-By: Claude Code <noreply@github.com>
3b8341c
This PR adds the transport of means and variances under a Kullback–Leibler constraint, the tools "the KL-to-Bernstein transport" and "the variance transfers [MDJKLV21, Lemmas 10–12]" of RF-Express: if
KL(μ ‖ ν) ≤ aandfhas values in[c, d], then|∫ f ∂μ - ∫ f ∂ν| ≤ √(2 Var_ν(f) a) + (d - c) a / 3,Var_ν(f) ≤ 2 Var_μ(f) + (8/3)(d - c)² aandVar_μ(f) ≤ 2 Var_ν(f) + (5/6)(d - c)² a. It also adds the two variance transports of [MDJKLV21, Lemma 12] that use no KL constraint.Roadmap: ReinforcementLearning
The target is ReinforcementLearning Layer 19.2 (RF-Express [MDJKLV21, Theorem 1]), whose list of tools includes "the KL-to-Bernstein transport: if KL(p,q) ≤ α and f ∈ [0,b], then |pf − qf| ≤ √(2Var_q(f)α) + (2/3)bα" and "the variance transfers [MDJKLV21, Lemmas 10–12]". The case
c = 0,d = bofabs_integral_sub_integral_le_of_klDiv_legives the roadmap's bound withb α / 3in place of2 b α / 3. After this PR, Layer 19.2 still needs counts versus pseudo-counts, Lemma 9 (∑_t u_{t+1}/max(1, U_t) ≤ 4 log(U_{T+1} + 1)), the inversion of Lemma 13 (open in #287), and then the algorithm with its PAC and sample-complexity theorem.The constants are sharper than in the sources, and I checked them by hand. [MDJKLV21, Lemma 10] (after [TM18]) bounds the positive root of
2α(V + D/3) − D² ≥ 0by√(2Vα) + 2α/3. Here, Chernoff's choicet = r / (v + b r / 3), withr = √(2 v a), in the sub-gamma boundexp x − 1 − x ≤ x² / (2(1 − x/3))gives√(2 v a) + b a / 3directly. This is the inversion from the high-probability Bernstein inequality [BLM13, Theorem 2.10] that LMLExplore already has inProbability/Moments/Bernstein.lean. The variance transports follow from it with(8/3)and(5/6)in place of4[MDJKLV21, Lemma 11], and the finite-space transport has the factor1/2in place of3[MDJKLV21, Lemma 12], because the constant(d − c)²/2can be subtracted under∑ (ν{x} − μ{x}) = 0. The one-sided transport only needsf ≤ ∫ f ∂ν + bν-a.e.,MemLp f 2 νandIntegrable f μ. The boundb ≥ 0follows from these, andb = 0is handled as a limit. The two-sided statements takefin any interval[c, d], not just[0, b].New declarations:
LMLExplore/InformationTheory/KullbackLeibler/Bernstein.lean(new file, namespaceLMLExplore):integral_sub_integral_le_of_klDiv_le(one-sided)abs_integral_sub_integral_le_of_klDiv_le[TM18], [MDJKLV21, Lemma 10]variance_le_two_mul_variance_add_of_klDiv_leandvariance_le_two_mul_variance_add_of_klDiv_le'[MDJKLV21, Lemma 11]t ≥ 0, the caseb > 0, and the specialisation tog ∈ [0, B]withVar_ν(g) ≤ B ∫ g ∂νLMLExplore/Probability/Moments/Variance.lean(namespaceProbabilityTheory):variance_le_integral_sub_sq:Var[X] ≤ ∫ (X − c)²variance_le_mul_integral_sub:X ∈ [c, d]givesVar[X] ≤ (d − c)(𝔼X − c), a weakening of Mathlib's Bhatia–Davisvariance_le_sub_mul_subvariance_le_two_mul_variance_add_mul_integral_abs_sub:|X − Y| ≤ DgivesVar[X] ≤ 2Var[Y] + 2D𝔼|X − Y|variance_le_variance_add_mul_sum_abs_sub: on a finite space,Var_ν(f) ≤ Var_μ(f) + (d − c)²/2 ∑|μ{x} − ν{x}|LMLExplore/Analysis/SpecialFunctions/Exp.lean:Real.exists_pos_le_mul_sqrt_add_sub_mul_exp_sub_one_sub, the closed-form inversion of the sub-gamma bound at Chernoff'st = r / (v + b r / 3). It is shared withmeasureReal_sqrt_add_le_sum_le_of_ae_leinProbability/Moments/Bernstein.lean, whose proof now uses it in place of its own copy of the inversion.LMLExplore/Analysis/Convex/Integral.lean:MeasureTheory.integral_mem_Icc_of_ae_mem_Icc, the mean of a function with values in[l, u]a.e. lies in[l, u]. The existingintegral_id_mem_Icc_of_ae_mem_Iccis now its special case.LMLExplore/Algebra/Order/Ring/Abs.lean(new file):sq_sub_mem_Icc_of_mem_Icc,x, y ∈ [c, d]gives(x - y) ^ 2 ∈ [0, (d - c) ^ 2].The proofs reuse:
TauCeti.ofReal_integral_sub_log_integral_exp_le_klDivMeasureTheory.MemLp.mgf_le_exp_mul_exp_sub_one_sub_of_ae_leandReal.exp_sub_one_sub_id_le_sq_divvariance_le_expectation_sq,variance_le_sub_mul_sub,Convex.integral_memandklDiv_eq_zero_iffNo declaration is removed or renamed.
Source: this PR ports item R23 of the paper project RemyDegenne/Essakine2026Tight, at commit 520dd39a55fa4e9a62a4a2d51f48163926176a5d (Apache-2.0). The source files are
Essakine2026Tight/ForLMLExplore/InformationTheory/KLBernstein.leanandEssakine2026Tight/ForLMLExplore/Probability/Moments/VarianceTransport.lean. Changes from the source:Real.poissonCramer,Real.expSubOneSub, item R26, which is not a ReinforcementLearning target), nor the AM–GM helpers ofAnalysis/Real/Sqrt.lean.[0, b]to[c, d], and the one-sided transport to functions bounded above.integral_mem_Icc_of_ae_mem_Icc(item R8) is added, under the same name, inAnalysis/Convex/Integral.lean.Probability/Moments/Variance.leanrather than a new file.Local checks (not CI):
lake build --iofailis green.Paper: RemyDegenne/Essakine2026Tight
Item: R23
🤖 Generated with Claude Code