Repository navigation
feat: the eventual regret of ucbF as the Gaussian instance of KL-UCB - #380
RemyDegenne wants to merge 2 commits into
Conversation
Co-Authored-By: Claude Code <noreply@github.com>
There was a problem hiding this comment.
🟡 proof-quality — request_changes claude/claude-opus-5-5
The new pull-count proof duplicates two pieces of Gaussian reasoning already in KLUCB.lean: the MGF dominance of each arm by the Gaussian family member, and the identity d(μ_a, μ*) = Δ_a² / (2σ²) from a rewrite chain. Both should become small lemmas shared with the existing proofs.
LMLExplore/Bandits/KLUCB.lean:799— Thehdomblock (lines 799-802) repeats thehdomblock ofklucbFam_gaussianReal_regret_eventually_le(lines 775-778), differing only inm = 0and a commonσ2. This hides a reusable fact: a sub-Gaussian arm is MGF-dominated onunivby the matching member of the Gaussian family. Fix: Add a lemma, e.g.HasSubgaussianMGF.mgfDominatedOn_expFamily_natParam_gaussianReal (m) (hσ2) : MGFDominatedOn id μ id ((gaussianReal m σ2).expFamily id ((gaussianReal m σ2).natParam id μ[id])) univ, and use it in both proofs.LMLExplore/Bandits/KLUCB.lean:809—rwa [expFamilyDiv_gaussianReal _ hσ2, ← neg_sub, neg_sq, ← hgap, inv_div]is a five-step rewrite chain. Its first four steps redo the per-arm computation insidesum_gap_div_expFamilyDiv_gaussianReal(line 759:rw [expFamilyDiv_gaussianReal _ (hσ2 a), ← neg_sub, neg_sq, ← gap_eq_bestArm_sub]), so the identityd(μ_a, μ*) = Δ_a² / (2σ²)is stated nowhere and is rederived twice. Fix: Factor outexpFamilyDiv_gaussianReal_eq_gap_sq (m) (hσ2) : (gaussianReal m σ2).expFamilyDiv id (ν a)[id] (ν (bestArm ν))[id] = gap ν a ^ 2 / (2 * σ2). Use it in the sum lemma and here, which turns this step intorwa [expFamilyDiv_gaussianReal_eq_gap_sq, inv_div]and also removes the auxiliaryhgap.
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 26s · 64.1k in (51.1k cache read, 13k cache write) / 2.4k out tokens · reviewing this diff · rubric ·
There was a problem hiding this comment.
Fixed in 7636fcb, as suggested: added HasSubgaussianMGF.mgfDominatedOn_expFamily_natParam_gaussianReal (for a general X/Q, mean a and generator m) and expFamilyDiv_gaussianReal_eq_gap_sq. Both hdom blocks are now one-liners using the first. The second replaces the rewrite chain in sum_gap_div_expFamilyDiv_gaussianReal, and in UCBF.integral_pullCount_eventually_le the last step is now rwa [expFamilyDiv_gaussianReal_eq_gap_sq _ hσ2, inv_div], without the auxiliary hgap.
AI review — changes requestedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
…CB proofs Factor out HasSubgaussianMGF.mgfDominatedOn_expFamily_natParam_gaussianReal and expFamilyDiv_gaussianReal_eq_gap_sq, used by sum_gap_div_expFamilyDiv_gaussianReal, klucbFam_gaussianReal_regret_eventually_le and UCBF.integral_pullCount_eventually_le. Co-Authored-By: Claude Code <noreply@github.com>
This PR derives the asymptotic part of [LS, Theorem 8.1] for the asymptotically optimal UCB algorithm
ucbF σ2as the Gaussian instance of KL-UCB for exponential families, as Bandits Layer 3.4 asks ("Deduce, as the Gaussian instance, the eventual form of Layer 2.2 for σ²-sub-Gaussian rewards"; Layer 2.2: "the eventual form is also the Gaussian instance of Layer 3.4"). Until nowucbF_regret_eventually_leandUCBF.integral_pullCount_eventually_leeach had their own ε-argument over the explicit finite-time bound. Now they follow fromklucbFam_gaussianReal(ucbF σ2is KL-UCB for the Gaussian family of varianceσ²):ucbF_regret_eventually_leisklucbFam_gaussianReal_regret_eventually_lewith a common variance.UCBF.integral_pullCount_eventually_leisKLUCBFam.integral_pullCount_eventually_lefor the Gaussian family, withd(μ_a, μ*) = Δ_a² / (2σ²)(expFamilyDiv_gaussianReal).Both keep their names and their exact statements. They move from
LMLExplore/Bandits/AsymptoticallyOptimalUCB.leantoLMLExplore/Bandits/KLUCB.lean, the first file where the KL-UCB family theorems exist;KLUCB.leanalready importsAsymptoticallyOptimalUCB.lean. Their old proofs are deleted. The explicit finite-time boundsUCBF.integral_pullCount_leanducbF_regret_lestay where they are, since Layer 2.2 says "the explicit constants above keep the proof through [LS, Lemma 8.2]". The module docstrings of both files are updated.AsymptoticallyOptimalUCB.leandrops its import ofLMLExplore.Bandits.Regret, which only the moved proof used. The only downstream user isBandits/AsymptoticOptimality.lean(UCBF.isConsistent,tendsto_ucbF_regret_div_log_gaussianBandit), which already importsKLUCBand builds unchanged.Public declarations removed or renamed: none. The two moved theorems keep their names and statements; only their module changes, from
LMLExplore.Bandits.AsymptoticallyOptimalUCBtoLMLExplore.Bandits.KLUCB.Milestone: Bandits Layer 3.4 (regret of KL-UCB for exponential families and its instances). After this PR, Layer 3 has only its completion examples left: the Bernoulli boundary level
U(0, c) = 1 - e^{-c}and a two-armed Bernoulli bandit.Verification (local checks, not CI):
tauceti-local-checks verify --policy ordinarypassed withscope: scoped: against 7fc78707241a5393aa85a7751e472cf1cb45fca9. The whole-librarylake build --iofailsucceeded (8551 jobs).local-axioms: checked 2 changed module(s), 64 declaration(s); 0 violation(s). Module-system, header-style and changed-module environment lint passed.Roadmap: Bandits
🤖 Prepared with Claude Code