Repository navigation
Add SZ richness selection, BSEL→SigmaPrj validation - #7
Merged
Merged
Conversation
New SigmaPrj operator (Costanzi 2026 Eq. 13 / errata E.3, background-free master equation): exact per-cell ring integration of the offset-NFW kernel via the R<->s symmetry, DeltaSigma_prj as its own kernel-swap integral (never a reconstruction from Sigma), law-of-cosines chord, parabolic photo-z window or hard top-hat, mass-weighted halo-centric truncation, and split z-grid pinned at the exclusion crossing. Exclusion is applied as the Costanzi counter term (default): inside the chord ball the correlated integrand is -1, cancelling the background's +1. Same total as deleting neighbours, but the background stays strictly uniform and the exclusion hole is booked in the cl channel, where a random-point-subtracted measurement keeps it. "ball"/"cl"/"none" remain as config switches. Default return channel is "cl": the two-halo term carries no background (cluster_toolkit convention); channel="sum" reproduces raw mass maps. Also: - selection/miscentering.py: fix two _query clamp bugs (x_mis below table range -> centred closed form; above range -> far-field sigma, ds = 0). - selection/bsel.py: XiNL(clip=), plateaus/marginalised_bias(b_eff=), operators_var, delta_stats, self-consistent lambda_tr kernel option. - Validation vs the Costanzi mock (validation/validate_sigma_prj_mock.py): 11/12 selected-to-random ratio bins within 0.02 in the 2h regime, absolute stack at 7.4%, parameter-free Delta_RND match 0.4-15%; bin representatives from the counts operator's own first moments. - Reproducible LaTeX report under papers/projection_lensing/ (9 pages, all values/figures generated from scripts). - Theory page docs/projection_lensing.md, notation.md section 5, validation.md, tagged getting_started cell, 15 new tests (875 pass). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Documentation-only restatement following the corrected excess-density equations (review/08282026/sigma_prj_excess_equations.md): exclusion acts on the complete pair distribution, so the pair weight of the random-point-subtracted observable is K_exc = (1 + b b_sel xi_NL) E - 1 -- the correlated term outside the chord ball, minus the background integrand (-w_rnd: no bias, no b_sel, no xi) inside it. The master equation in the module docstring, docs/projection_lensing.md, and the report's Section 1 now carry K_exc explicitly, with the channel-weight split (b_sel w_cl outside, w_ct = -w_rnd|ball inside), the 1/(2pi) normalization statement of the azimuthal average, and the angular-cap reading of the 3-D exclusion ball. No numerical change: exclusion="counter" already computed exactly this -- (1+x)E-1 == xE-(1-E) -- verified to machine precision (cl(counter) == cl(slab) - [rnd(uniform) - rnd(ball)], max rel err 1.4e-16). 875 tests pass unchanged; report rebuilt (9 pages). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Addresses the exclusion-discontinuity and resolution findings of the 2026-08-29 review (review/08282026/sigma_prj_excess_equations.md): - theta_edges(zob, lob): the exclusion boundary is a curve in the (theta, z) plane -- theta and z are coupled through the chord -- so the cell edges are now aligned with every z node's exact crossing angle theta_ex(z_k). No kernel cell straddles the indicator for any quadrature node; downstream shapes are dynamic. - DeltaSigma smooth term: the aperture-mean 2*pi*s*SigmaBar_mis(<R|s) curves on the r_s scale near s ~ R; the per-cell edge trapezoid was -5.5% at R=8 (test resolution). Replaced with per-cell 4-node Gauss-Legendre; ds now matches the direct SigmaBar(<R) - Sigma(R) cross-check to +0.2% (new test). - Defaults n_theta 64->96, n_z_side 40->160, justified by a measured per-axis convergence scan (in the constructor docstring): mass converged at 40; theta 96 puts Sigma within 0.3%; the exclusion step in the z direction converges ~first order and needs n_z_side=160 for DeltaSigma near R_excl (+9% at 40, +0.14% at 160 vs the n=640 reference). z cost is weights-only, so the high default is cheap. - Docstrings: rnd is the selected-halo background column (mass cut, untruncated wings, aperture-dependent), not the cosmological mean; R_excl's neighbour-mass independence named as the Costanzi effective prescription (Garcia et al. 2021 for the physical kernel). - Demo and getting_started example switched off the exclusion="ball" + default channel="cl" trap (that combination silently omits the exclusion deficit); counter is the mode-invariant excess. - New tests: counter cl == sum - uniform rnd and == slab cl - ball hole; DeltaSigma vs direct excess. 876 pass. - papers Makefile: multi-output recipes via stamp files (a plain multi-target rule re-runs the heavy scripts once per target under make -B). Mock validation re-run with the new numerics: 11/12 ratio bins (same single failing bin), absolute leg 7.3%; report and docs figures regenerated (9 pages). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
For embedding in issue #5: the 12-bin ratio grid and the absolute stacked-column comparison, produced by validation/validate_sigma_prj_mock.py after the physical-units refactor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mock cosmology reassigned (the mock is DES Y3 data, not Buzzard): Planck 2018 assumed, and the MOR switched to HodMor.from_lognormal(). See issue #5 for the leg-by-leg comparison. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rlap validation Rewrites halo_mass_function, sigma, bias, and projection to share common m/z/r grids; adds los_integrals utility and validation scripts for bsel quadrature, MOR notebook, and survey area overlap; restructures docs (observables.md split into number_counts/shear_proj/stacked_shear). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…alo model SelBiasEngine now reuses SigmaPrj's LosGeometry/integrate_los machinery (utils/los_integrals.field_integrand shared between both) instead of its own hand-rolled z-ring+outer grid, and takes sigma_prj= to share one built halo model instead of rebuilding hmf/bias/xi_nl and hand-wrapping PhysicalMassMor at every call site. I2-I1 is now a direct stable quadrature (D) instead of a float subtraction, removing the cancellation risk at the closure's known instability point; b_eff switched from trapezoid to Gauss-Legendre; plateaus() renamed to b_small_large(). GL node counts bumped with margin across both engines (bsel n_theta particularly: the new theta-outer geometry needed far more resolution than the paper's per-z split did) and checked against an independent scipy.quad reference and self-convergence sweeps -- both converge monotonically, final agreement 0.04-0.09%. Updated every downstream call site to the new API (demo, tests, both validation scripts, figure generators, the example notebook source, and the projection_lensing paper's make_results.py/make_inner_study.py), and trimmed the bsel.py/projection.py module docstrings to the project's 15-line budget, moving the derivations into docs/selection_bias.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…w validation report _ltr_weights' lambda_tr posterior fed the closure a mean excess richness 1.5-2.2x too high (an exponential-tilt divergence against a steep richness function), which the closure's own 18-40x gain amplified into a 3-4x error on b_small. Replaced with excess_delta/gamma_lambda, a parameter-free estimate from the model's own operators (docs/plan-bsel-stable-closure.md). Median residual against the digitized Costanzi et al. (2026) Fig. 6 curve drops from ~50% to ~10%; mock catalogue leg D goes to 12/12 bins passing. Also: confirmed Buzzard v1.1 cosmology (was an unconfirmed Planck-2018 guess) in the mock validation harness; disk caching for TinkerMassFunction/BiasModel mirroring PkGrid's; SelectionBiasTable no longer loses damping/theta0_frac on save/load; new papers/validations/ report (Fig.6 + mock validation, numerical methods, alpha and cosmology sensitivity). Open and documented, not fixed: the closure's redshift dependence has the wrong sign (plan-bsel-stable-closure.md sec 9), and SigmaPrj's aperture transition sits ~2x too far out in R vs the digitized curve, narrowed to MassShells but unresolved (plan-sigma-prj-shell-shift.md). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
excess_delta + fixed sigmoid shape (damping=2.5, theta0_frac=0.5) both miss Matteo Costanzi's own reference curve across all 4 independent richness bins; a plain power law (b_sel/b_eff = plateau + A*R^p, p~-1 to -1.4) matches every bin to R2>0.92, far better than the paper's own sigmoid form at any parameterization tried. See GitHub issue #6. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…lanning docs - New SZ-richness bias module (selection/sz_richness.py) - BSEL closure validation: reproduces SigmaPrj via closure test - Archive 5 planning docs to docs/_archive, delete obsolete plan - Refresh projection_lensing.md theory section - Minor tweaks: HMF caching, BSEL excess-richness estimate Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
…kips - miscentering_kernel.py was missing `from __future__ import annotations`, so its `tuple[...]` annotations raised TypeError on Python 3.8 at collection time (PEP 585 generics need 3.9+ without the future import) - flake8 wasn't in dev deps even though the workflow calls it - Cleared F811 (duplicate imports), F401 (unused imports), E305/E266 style errors flagged by flake8's E9/F63/F7/F82 gate - test_pkgrid.py's pyccl/camb-backed tests now skip via pytest.importorskip instead of failing when those optional backends aren't installed (matches the documented "compare" extra convention) Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
estevesjh
force-pushed
the
feature/projection-lensing
branch
from
September 15, 2026 06:50
ed2447e to
c26531b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
pytest tests/python validation/validate_bsel_reproduces_sigma_prj.pymake -C docs html🤖 Generated with Claude Code