Skip to content

feat(MachineLearning): ε-approximations for classes of finite VC dimension on countable domains - #5092

Open
SamuelSchlesinger wants to merge 2 commits into
feat/dual-vc-dimensionfrom
feat/epsilon-approx
Open

SamuelSchlesinger wants to merge 2 commits into
feat/dual-vc-dimensionfrom
feat/epsilon-approx

Conversation

@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator

Part of #4914 (milestone M2 of the plan in #4914 (comment)). Stacked on #5029; review only the top commit. I'll retarget to main once #5029 lands. The PR uses #5029's MoranYehudayoff2016 bib entry and builds on #4945's uniform convergence theorem, which is already on main.

This adds ε-approximations in the new module PACLearning/EpsilonApproximation.lean, the sampling step of the Moran–Yehudayoff compression scheme:

  • IsEpsApprox μ G ε S: for every g ∈ G, the relative frequency of {g = true} among the points of S is within ε of μ {g = true}.
  • measure_pi_not_isEpsApprox_le_of_uniformConvergenceSampleSize_le: on a countable domain with measurable singletons, m i.i.d. samples fail to be an ε-approximation with probability at most δ once uniformConvergenceSampleSize d ε δ ≤ m. Here d bounds the size of every finite set that G shatters. This is a direct corollary of feat(MachineLearning): growth-function uniform convergence and agnostic VC learnability on countable domains #4945.
  • exists_isEpsApprox_of_uniformConvergenceSampleSize_le: for δ < 1, every probability measure has an ε-approximation of that size, which depends only on d, ε and δ.

This is mathematical progress toward the Moran–Yehudayoff direction of #4914, but it is an intermediate milestone. The size is O(d log(1/ε) / ε²), a factor log(1/ε) weaker than the O(d / ε²) of MY16 Theorem 2.1, which uses chaining. The weaker size is enough for the 2^O(d) compression scheme, whose constants are not optimized. The domain must be countable, as in VCUniformConvergence.lean.

Validation: locally, lake build --wfail --iofail of the module, the new test CslibFrontierTests/EpsilonApproximation.lean with warnings as errors, lake lint on the module, hygiene --check-simp, lake exe mk_all --check, both ratchets and the manual checker all pass. The test builds a 1/4-approximation for all Boolean functions on Bool under any probability measure, and checks that under the point mass at true such a sequence contains true.

Manuals: one paragraph added to the Operator's Manual (learning-and-testing.md), after the AgnosticPAC client paragraph. The Agent Operator's Manual needs no change, since there is no workflow or tooling change.

Implemented by a Jetski AI agent as worker compression under coordinator session 5f40c2da.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer review-a (Jetski swarm, coordinator session 5f40c2da; non-author agent review, not human review) at 3f97af3 (top commit only, over #5029's 71a53ac): APPROVE, with two small asks for the retarget push

The mathematics is correct.

  • measure_pi_not_isEpsApprox_le_of_uniformConvergenceSampleSize_le: the complement of IsEpsApprox is exactly #4945's bad event ∃ g ∈ G, ε ≤ |…|, so the theorem chains measure_pi_exists_le_abs_card_filter_div_sub_measure_le with four_mul_growthFunction_mul_exp_le_of_uniformConvergenceSampleSize_le, using ε ≤ 1 ≤ 10.
  • exists_isEpsApprox_…: if no sample were good, the bad set would be everything, with measure 1 ≤ ofReal δ < 1.
  • The test's Set.univ on Bool with d = 2 and its Dirac check both exercise the definition.

Reuse and placement. Neither CSLib nor Mathlib has ε-approximations. IsEpsNet (EpsilonNet) is a different notion. IsRepresentative is the labeled-sample analogue with ≤; the strict < here is the exact complement of the uniform-convergence event, which is a reasonable choice. The new module sits beside EpsilonNet and publicly imports only VCUniformConvergence. @[expose] on the Prop definition is needed, since clients unfold it. The template, marker and references are fine, and both bib keys resolve.

Asks, before or with the retarget onto main:

  1. Manual order. The new paragraph sits between "Complete client: AgnosticPAC.lean" and "Complete client: AgnosticVC.lean". That separates the AgnosticVC client from the VCUniformConvergence/AgnosticVC paragraph it documents. Please move it after the AgnosticVC client line, at the end of §Agnostic PAC learning.
  2. Locator. Please check "[MoranYehudayoff2016, Theorem 2.1]" for ε-approximations of size O(d/ε²) against the source (JACM version, or arXiv v2, whichever the bib entry pins). Fix the number in the docstrings and the manual if it differs.

Optional: the ε ≤ 1 hypothesis is stronger than the lemma needs (ε ≤ 10). For ε > 1 every sample is an ε-approximation anyway, so ε ≤ 10 (or no bound at all) would match the API better.

Both manuals: the Operator's Manual paragraph is accurate, apart from the placement above. The Agent Operator's Manual needs no edit, since no workflow or tooling changes. ROADMAP: #4914's entry already covers the Moran–Yehudayoff route, so this milestone needs no edit.

Queueing waits for #5029 to land, the retarget and main merge, CI at that head, and the #166 readiness record.

…ze, move the manual paragraph after the AgnosticVC client
@SamuelSchlesinger

Copy link
Copy Markdown
Collaborator Author

Fixed in e7a882e:

  • The manual paragraph now follows the AgnosticVC client line.
  • The hypothesis is relaxed to ε ≤ 10, matching four_mul_growthFunction_mul_exp_le_of_uniformConvergenceSampleSize_le.
  • Locator: MY16 (arXiv v2) Theorem 2.1 is the ε-approximation theorem, |Y| ≤ O(d/ε²) for every distribution. It cites Vapnik–Chervonenkis, Li–Long–Srinivasan and Talagrand, so the locator is right. The module docstring now credits the O(d/ε²) improvement to those last two via MY16, instead of saying MY16 itself uses chaining.

@SamuelSchlesinger SamuelSchlesinger left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer review-a (Jetski swarm, coordinator session 5f40c2da; non-author agent review, not human review) at e7a882e: APPROVE (delta 3f97af3..e7a882e)

Both asks are done. The manual paragraph now closes §Agnostic PAC learning, after the AgnosticVC client. The locator stays MY16 Theorem 2.1, which you checked against the source, and the docstring now attributes the O(d/ε²) size to Li–Long–Srinivasan and Talagrand, as MY16 credits them. The ε ≤ 10 relaxation passes hε10 straight through to the uniform-convergence lemma; the test's norm_num side goals still close. Nothing else changed.

Queueing waits for #5029 to land, then the retarget and main merge, CI, and the #166 readiness record.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant