Skip to content

Region-scoped observation pooling as bundle data: lnk_species_pooling() (#290) - #291

Merged
NewGraphEnvironment merged 11 commits into
mainfrom
290-region-scoped-dv-bt-observation-pooling
Sep 28, 2026
Merged

NewGraphEnvironment merged 11 commits into
mainfrom
290-region-scoped-dv-bt-observation-pooling

Conversation

@NewGraphEnvironment

@NewGraphEnvironment NewGraphEnvironment commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Observation pooling is data now. Which observation species count as evidence for which model species moves out of code into a bundle tracker, configs/default/species_pooling.csv: one dated, sourced row per decision, scoped to a region, sub-region or WSG. Either side can be a group from species_groups.csv.
  • The resolver is species-agnostic. New exported lnk_species_pooling() knows no species: DV into BT, or all salmon together, is a row.
    • The most specific scope wins, then the row naming fewer groups. Unresolved ties error, and anything unlisted is not pooled.
    • Regions come from the new package-level inst/extdata/wsg_regions.csv, which uses each group's outlet wscode and is built by data-raw/wsg_regions.R. It is also hashed into config_hash for any bundle that declares a tracker.
  • The consumers read the tracker. lnk_habitat_validate() accepts the per-WSG table as species_obs; the list form and its default are unchanged. data-raw/habitat_validate.R resolves pooling once and applies it to both bundles. The Research: calibrate CH and BT gradient and channel-width thresholds from observations #284 obs producer joins the table in place of its hard-coded DV CASE. Both drivers stop if the pooling bundle's presence table disagrees, WSG by WSG, with the scored bundle's.
  • Tracker: DV → BT in the Fraser, Mackenzie, Columbia/Kootenay, and the Skeena only above Hazelton. The region seed is the operator's call of 2026-09-26; the Skeena split is their call of 2026-09-27.
  • Evidence: DV/BT naming through time and the pooling sensitivity are measured by data-raw/species_pooling_evidence.R, with maps and figures. The write-up is research/species_pooling.md, and the walkthrough page is When a Dolly Is a Bull Trout (private).

Related Issues

Test plan

  • devtools::test() (at 4e9893d): 2151 pass, 3 fail. The 3 are all the :63333 bcfishpass tunnel being down (test-lnk_db_conn.R:10, test-lnk_wsg_resolve.R:143, :154), in files this branch does not touch.
  • New tests:
    • 43 expectations in test-lnk_species_pooling.R, on synthetic codes (AAA/BBB/GRP), so nothing passes on BT/DV by accident;
    • the data-frame species_obs form, end to end against the list form;
    • a mocked test that a regions edit moves config_hash;
    • a check that every new dictionary consumed_by ref lands on a line naming its column.
  • Guards proven by mutation in scratch copies: scope precedence, the taxon tie-break, pool = no, the presence gate, the df branch, the hash block, and wrong dictionary refs each turn their tests red.
  • Under the region rule alone, the Research: calibrate CH and BT gradient and channel-width thresholds from observations #284 obs producer re-ran byte-identical in every CSV and PNG except the province-wide ledger step 3 (DV 8,888 → 6,138). That number was predicted before the run, and independently by review.
  • Under the Hazelton split, the committed Research: calibrate CH and BT gradient and channel-width thresholds from observations #284, Validate modelled habitat against fish observations per species and watershed group #283 and scenario evidence were regenerated from the committed code (911bed3).
  • The Validate modelled habitat against fish observations per species and watershed group #283 validation baseline (default vs bcfishpass, 55 + 59 WSGs, knowledge @ 508bf44) re-ran byte-identical under the region rule. Under the split, its shared BT set falls from 4,600 to 3,771 locations, CH is unchanged, and the committed CSVs are updated.
  • Negative check with Skeena pool = no: exactly 1,929 DV rows drop, in the 9 Skeena WSGs; nothing else moves.
  • devtools::document() and pkgdown::check_pkgdown() clean.
  • lintr: not clean.
    • New code adds hanging-indent style lints, the same pattern as the repo backlog (67 in test-lnk_log.R on main).
    • Plus 3 object_usage_linter false positives for helpers that are absent from the stale installed package.

Notes

🤖 Generated with Claude Code

https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx

NewGraphEnvironment and others added 6 commits September 26, 2026 16:55
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
inst/extdata/wsg_regions.csv assigns each of the 246 groups the region its
outlet drains to (first wscode segment) and, where curated, a sub-region
(longest whole-segment prefix). Kootenay (300.625474) is the first.
Coastal and cross-border region names are drafts, marked in the defs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
Which observation species count as evidence for which model species is
now a tracker in the bundle (species_pooling.csv), scoped to region,
sub-region or WSG through inst/extdata/wsg_regions.csv, with named groups
(species_groups.csv) usable on either side. The resolver knows no species:
DV into BT, or all salmon together, is a row. Most specific scope wins,
then the row naming fewer groups; unresolved ties error; anything
unlisted is not pooled.

default declares both (seeded: DV -> BT in Fraser, Mackenzie, Skeena,
Columbia, Kootenay); default_tuned inherits them. config_hash now also
covers wsg_regions.csv for bundles with a tracker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
#290)

lnk_habitat_validate() accepts lnk_species_pooling()'s per-WSG table as
species_obs (checked as a data frame first; the list form and its default
are unchanged). habitat_validate.R resolves pooling once, from the first
bundle that declares a tracker (--pooling=), applies it to both bundles
and stamps it. query_habitat_thresholds_obs.R joins the resolved table in
place of the hard-coded DV CASE. Both refuse a pooling bundle whose
presence differs, WSG by WSG, from the scored one's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
research/species_pooling.md holds the tracker's state per region. The #284
evidence is refreshed: only the province-wide ledger step 3 moves (DV
8,888 -> 6,138); every other CSV re-runs byte-identical, as does the #283
validation baseline. Method lines in the research docs now point at the
tracker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
NewGraphEnvironment and others added 5 commits September 26, 2026 21:55
Interior DV is a legacy name (the DV share of char records falls from ~90%
before 1990 to under 15% after 2000). Skeena DV is not: 86% or more in every
decade, and 85% of Skeena DV streams were re-sampled after 1995 and still
recorded only DV. default_tuned's BT rear_gradient_max of 0.1349 depends
on the Skeena DV records: without them the rule gives 0.1249, the BT-only
value. The above-Hazelton cut keeps 0.1349, by 0.0009.

data-raw/species_pooling_evidence.R runs four scenarios (S3, pre-1995
interior pooling, computed from the S0 evidence and checked against the
producer on S0-S2) and scores S1/S2 through the validator. Figures and
maps are in data-raw/logs/species_pooling_290/; write-up in
research/species_pooling.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…290)

Skeena DV is a current name, not a legacy one, so the Skeena region row
now keeps DV apart and a new sub-region, 'Skeena above Hazelton' (BULK,
MORR, KISP, BABL, BABR, SUST, MSKE, USKE), pools it. wsg_regions_defs.csv
gains explicit-membership sub-regions, since MSKE and USKE outlets sit on
the Skeena mainstem like LSKE and no wscode prefix can split them.

species_pooling.csv gains an optional obs_year_max: a row then pools only
records dated in or before that year (undated records do not qualify).
lnk_species_pooling() carries the winning row's limit; the validator and
the #284 obs producer apply it per record, and the validator refuses a
year limit on a source with no observation_date. Empty in the seed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
The reimplementation applied a year limit to already-deduplicated S0
evidence, while the producer applies it per record before deduplicating,
so a location with DV records on both sides of 1995 was judged by
whichever record survived (review round, 2026-09-27). Every scenario now
reports its own producer run; the reimplementation cross-checks the
scenarios without a year limit and supplies BT_only. The S3 figures
published earlier came from the reimplementation and are superseded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
…n split (#290)

Under the adopted tracker 852 Skeena DV records below Hazelton drop out
of the BT evidence. #284: pooled BT rearing gradient P95 0.1348 -> 0.1309,
n 4,764 -> 3,988; the rule still gives 0.1349, now by 0.0009, and no
verdict moves. #283: the shared BT set is 3,771 locations (was 4,600); CH
is byte-identical. The scenario table now reports producer runs for all
five scenarios (S3 corrected: 543 DV records, P95 0.1202) and adds S4,
the split with the interior limited to pre-1995 records (P95 0.1310).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx
@NewGraphEnvironment
NewGraphEnvironment merged commit e86cfc8 into main Sep 28, 2026
1 check passed
@NewGraphEnvironment
NewGraphEnvironment deleted the 290-region-scoped-dv-bt-observation-pooling branch September 28, 2026 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Region-scoped DV→BT observation pooling, tracked in bundle CSVs

1 participant