Repository navigation
feat(MachineLearning): the dual class and Assouad's dual VC bound - #5029
SamuelSchlesinger wants to merge 2 commits into
Conversation
Add PACLearning/DualVCDimension.lean: dualClass C (the evaluations c ↦ c x on the concepts of C), Assouad's bound (shattered families of the dual class have at most 2^(d+1) - 1 members when C's shattered sets have at most d points), reduced to Mathlib's Finset.exists_shatters_of_dualFamily_shatters, and the precomposition lemmas SetShatters.injOn_of_comp / image_of_comp / card_le_of_comp used by the Moran–Yehudayoff compression scheme. First milestone of the remaining objective of #4914.
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Reviewer review-a (Jetski swarm, coordinator session 5f40c2da; non-author agent review, not human review) at f5b8120: APPROVE
What is proved. card_le_of_setShatters_dualClass: if every finite set shattered by C has at most d points, every finite family shattered by dualClass C has at most 2^(d+1) − 1 members, so vcDim (dualClass C) ≤ 2^(vcDim C + 1) − 1. This is Moran–Yehudayoff's Claim 1.5 (VC(C*) < 2^(d+1)), due to Assouad.
Proof. The proof reuses Mathlib's Finset.exists_shatters_of_dualFamily_shatters instead of re-proving the coding argument. I checked each step:
- One witness point is chosen per subfamily of
F. Tracing on those points is injective onF, because the witness for{c}separatesc. - The dual family of the traces shatters the traces: for a subfamily
t, the witness of{c ∈ F | τ c ∈ t}cuts outt. - If
|F| ≥ 2^(d+1), Mathlib returns a shattered(d+1)-subsetTof the witnesses. Every trace is a concept restricted toX ⊇ T, soCshattersT, which is a contradiction.
Going through the bitstring lemma instead of vcDim_dualFamily_le avoids having to bound the finset VC dimension of the trace family, so this is the shorter route. The precomposition lemmas are also correct. injOn_of_comp takes the singleton {i}, which some g ∘ φ cuts out, so φ j = φ i forces j = i. image_of_comp pulls V ⊆ φ '' W back to W ∩ φ ⁻¹' V.
Reuse and placement. Mathlib has the finset-level bound. CSLib, this library on main and the queued #4939 have no dual class, and nothing else here gives a SetShatters image/injectivity transfer. The new module is a small topic module with a proof-only import of Mathlib.Combinatorics.SetFamily.DualVC. dualClass is unexposed, and clients use mem_dualClass, which is right. The test is non-vacuous: it checks the bound for the thresholds on ℕ and a family that the dual class does shatter.
Sources and manuals. Claim 1.5 matches arXiv:1503.06960. The Assouad1983 and MoranYehudayoff2016 bib entries match the published venues. The Operator's Manual paragraph describes the actual API. The Agent Operator's Manual needs no edit.
Optional nits, not blocking:
SetShatters.injOn_of_comp,image_of_compandcard_le_of_compare general shattering facts that don't involve duals. They could sit next toSetShatters.card_le_vcDiminPACLearning/VCDimension.lean, so that classes defined by precomposition can find them without importing the dual module. Keeping them here is fine as long as M3 is their only client.- Mathlib's DualVC docstring gives Assouad's locator as Theorem 2.13. If that is checked, it could be cited here too.
Queueing waits for ci-checks and its notices.
|
The merge queue marks f5b8120 UNMERGEABLE (position 14). It now conflicts with |
# Conflicts: # manual/circuits-logic-and-other-models/learning-and-testing.md
|
The queue entry at f5b8120 went UNMERGEABLE after #4939 and #4945 landed (conflict in |
SamuelSchlesinger
left a comment
There was a problem hiding this comment.
Reviewer review-a (Jetski swarm, coordinator session 5f40c2da; non-author agent review, not human review) at 71a53ac: APPROVE (merge delta only)
This is a merge-resolution review of the approved f5b8120 merged with main at 60058cd, after #4939 and #4945 landed.
scripts/check_restack.py f5b81206a8 71a53ace46 reports a replay conflict only in manual/circuits-logic-and-other-models/learning-and-testing.md. In the GrowthFunction paragraph, main's current text (including #4945's growthFunction_le_exp_mul_pow clause) is kept verbatim as the prefix. This PR's appended DualVCDimension sentences are byte-identical to the approved ones. Main's new Halfspaces paragraph follows, and no conflict markers remain. Every non-manual file's change against main is identical to the approved change against its base, with the same totals (5 files, +266/−1). The head also merges cleanly with the current main (580d9c9).
Both manuals: the Operator's Manual change is the approved text in its new context. The Agent Operator's Manual is not affected.
I'll requeue once ci-checks passes at this head and the merge-only readiness note is linked.
|
Readiness record for
|
|
Queue record (review-a, Jetski swarm, coordinator session 5f40c2da; agent record, not human review): queued |
Adds the dual class of a binary concept class and Assouad's dual VC bound, the first milestone of the Moran–Yehudayoff direction of #4914 (plan).
New module
PACLearning/DualVCDimension.lean:dualClass C : ConceptClass ↥C Bool, the evaluationsc ↦ c x, i.e. the columns ofC's concept-by-point matrix.card_le_of_setShatters_dualClass: if every finite set shattered byChas at mostdpoints, every finite family shattered bydualClass Chas at most2 ^ (d + 1) - 1members.hasFiniteVCDim_dualClassandvcDim_dualClass_legive the same bound forvcDim.SetShatters.injOn_of_comp,SetShatters.image_of_compandSetShatters.card_le_of_comp: if{g ∘ φ | g ∈ G}shattersW, thenφis injective onWandGshattersφ '' W, soG's size bound transfers. The compression scheme (M3 of the plan) applies this to the dual class pulled back along "the hypothesis learned from a block of indices".Mathlib already proves Assouad's bound for finite set families (
Finset.vcDim_dualFamily_le). This PR does not re-prove it. It reduces to Mathlib's bitstring-coding lemmaFinset.exists_shatters_of_dualFamily_shatters: it picks one witness point for each subfamily of a shattered familyF, traces the concepts ofFon those points (the trace is injective onF), shows that the dual family of the traces shatters the traces, and turns the shattered(d + 1)-set Mathlib returns into one shattered byC. No existing library, CSLib or Mathlib declaration defines a dual concept class or the precomposition lemmas.Progress type: mathematical (concept-class form of Assouad's bound) and infrastructure for the compression scheme.
Sources
VC(C*) < 2^(d+1). I verified the locator against arXiv:1503.06960v2. JACM numbering may differ, as the bib note records.Validation
lake build --wfail --iofail CslibFrontier.MachineLearning.PACLearning.DualVCDimension, the new testCslibFrontierTests/DualVCDimension.leanwith warnings as errors,lake exe mk_all --check,lake linton the module,scripts/check_hygiene.py --check-simp, the encoding readability and projection-chain ratchets, andscripts/check_operator_manual.py. The test checks the bound for the thresholds onℕ(VC dimension1, so at most3shattered thresholds andvcDim (dualClass thresholdClass) ≤ 3), and that the dual class shatters a one-threshold family. Full validation: CI.Manual impact
references.bib: addsAssouad1983andMoranYehudayoff2016.Part of #4914.
Implemented by a Jetski AI agent as worker compression under coordinator session 5f40c2da.