From a4f01130ca6d8f29a9f12fab15804b44cf666ddb Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 22:54:13 -0700 Subject: [PATCH 1/9] Initialize PWF baseline for #286 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- planning/active/findings.md | 34 ++++++++++++++++++ planning/active/progress.md | 8 +++++ planning/active/task_plan.md | 69 ++++++++++++++++++++++++++++++++++++ 3 files changed, 111 insertions(+) create mode 100644 planning/active/findings.md create mode 100644 planning/active/progress.md create mode 100644 planning/active/task_plan.md diff --git a/planning/active/findings.md b/planning/active/findings.md new file mode 100644 index 00000000..8b7fcf09 --- /dev/null +++ b/planning/active/findings.md @@ -0,0 +1,34 @@ +# Findings — Thread fresh params_method (per-WSG cw/mad) through lnk_pipeline_classify (#286) + +## Issue context + +**If done:** link configs can put watershed groups on bcfishpass's discharge (`mad`) habitat model and have it take effect. **If never:** every link run classifies on channel width, whatever a config's `parameters_habitat_method.csv` says. Nothing breaks either way, since the fresh default is the bundled all-`cw` table. + +fresh v0.35.0 (NewGraphEnvironment/fresh#220, PR #224) added `params_method` to `frs_habitat_classify()` and `frs_habitat()`: a data frame with `watershed_group_code` and `model` (`cw` or `mad`). `lnk_pipeline_classify()` calls `frs_habitat_classify()` by name without it (`R/lnk_pipeline_classify.R:95`). The config's method CSV needs to be loaded (`lnk_config()`?) and passed through. + +Under `mad`, fresh follows bcfishpass in three ways. Species with no MAD thresholds get no stream habitat from inheriting rules. Rule-level `channel_width` (the river-polygon bypass) is ignored. SK/KO lake rearing is based on polygon membership. fresh does not implement bcfishpass's `stream_order >= 8` spawning bypass, which is worth knowing when comparing against bcfishpass for `mad` groups. + +### Comment (2026-10-01) + +Literature input for the `cw` vs `mad` choice: we reviewed 17 Pacific Northwest intrinsic-potential models +(Oregon, Washington, California, Alaska and BC; Sheer et al. 2009 Table A1 and later work) for how each one +sized streams. + +- **None thresholds on gauged flow.** Wherever mean annual flow is used (Burnett 2007, Agrawal 2005, Lindley + 2006 and others), it is a regression of gauge flows on drainage area and mean annual precipitation. +- **Five models threshold directly on modelled channel width.** Cooney & Holzer 2006, for example, regress + field widths on drainage area and precipitation. +- **The exception is BC's own MAD.** Rebellato et al. 2024 use a hydrologic model where modelled MAD exists, + and modelled width as the discharge surrogate everywhere else. + +So outside existing MAD coverage, building MAD by regression would repeat the width model's inputs. The open +question is whether the process-based MAD carries information width does not, for example in snowmelt- and +glacier-fed basins. That is measurable: compare `cw` and `mad` classification where both exist. + +Thresholds in both currencies, with verbatim quotes, are in NewGraphEnvironment/knowledge#30 (`research/ip_models.md`). + + +## Errors Encountered + +| Error | Resolution | +|-------|------------| diff --git a/planning/active/progress.md b/planning/active/progress.md new file mode 100644 index 00000000..6b13b726 --- /dev/null +++ b/planning/active/progress.md @@ -0,0 +1,8 @@ +# Progress — Thread fresh params_method (per-WSG cw/mad) through lnk_pipeline_classify (#286) + +## Session 2026-10-01 + +- Plan-mode exploration — phases approved by user (method CSV per bundle; mad_m3s working-only; provenance via config_hash only) +- Created branch `286-thread-fresh-params-method-per-wsg-cw-ma` off main +- Scaffolded PWF baseline from issue #286 with approved phases +- Next: start Phase 1 diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md new file mode 100644 index 00000000..9918800e --- /dev/null +++ b/planning/active/task_plan.md @@ -0,0 +1,69 @@ +# Task: Thread fresh params_method (per-WSG cw/mad) through lnk_pipeline_classify (#286) + +**If done:** link configs can put watershed groups on bcfishpass's discharge (`mad`) habitat model and have it take effect. **If never:** every link run classifies on channel width, whatever a config's `parameters_habitat_method.csv` says. Nothing breaks either way, since the fresh default is the bundled all-`cw` table. + +## Context + +fresh v0.35.0 (fresh#220) added `params_method` to `frs_habitat_classify()`, a data +frame of `watershed_group_code` and `model` (`cw`/`mad`). link calls it without that +argument (`R/lnk_pipeline_classify.R:95`), so every run classifies on channel width +whatever a config wants. Exploration found three things the issue body does not say: + +- **No bundle carries a method CSV.** "The config's method CSV" does not exist yet. +- **Working streams have no `mad_m3s`.** `.lnk_pipeline_prep_network()` + (`R/lnk_pipeline_prepare.R:689`) joins only `channel_width`. fresh stops before any + write when a `mad` group's table has no `mad_m3s`. The source is + `whse_basemapping.fwa_stream_networks_discharge`, keyed on `linear_feature_id`. Local + fwapg holds 2.7M rows across 150 WSGs, and 2.0M of them have a non-NULL `mad_m3s`. +- **link pins `fresh@v0.33.0`.** That release has no `params_method`. The latest is + v0.36.2. v0.36.0's `frs_db_conn()` change reaches link only through + `data-raw/wsg_vignette_data.R:204`. + +All three bundle thresholds CSVs already carry `*_mad_min`/`*_mad_max`, so nothing +needs adding there. + +**Decisions made at this gate:** +1. Every base bundle declares `parameters_habitat_method.csv`, the #282 pattern. A + custom bundle that declares none falls back to fresh's copy, which is hashed into + `config_hash` by a fixed name. +2. `mad_m3s` goes on the **working** streams table only. The persist shape does not + change. +3. Provenance comes through `config_hash` only. No new log table and no new log column. + +## Phase 1: fresh pin + capability guard +- [ ] DESCRIPTION: `fresh (>= 0.35.0)`, `Remotes: NewGraphEnvironment/fresh@v0.36.2`. Reinstall fresh at the tag and confirm `params_method` is in `formals(fresh::frs_habitat_classify)` +- [ ] `lnk_preflight_fresh()` (`R/lnk_preflight_fresh.R`): assert the `params_method` formal on `frs_habitat_classify` (a capability, not a version). Add a test that restores the defect and shows the guard fires +- [ ] Check `data-raw/wsg_vignette_data.R`'s bare `fresh::frs_db_conn()` against the 0.36.0 behaviour change, and note the result in findings + +## Phase 2: method table as bundle data +- [ ] Add `parameters_habitat_method.csv` to `bcfishpass`, `default`, `default_extrabreaks` and `default_rearbreaks`: a frozen copy of `smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv` (188 groups, all `cw`). Diff it against fresh's bundled copy first and record the result. `default_tuned` inherits it through `extends` +- [ ] For each, add a `files: parameters_habitat_method:` entry and a provenance block (source, upstream_sha, synced, checksum, shape_checksum), the same as the thresholds entries +- [ ] Add a `.lnk_habitat_method_csv(cfg)` resolver in `R/lnk_config.R`, next to `.lnk_habitat_thresholds_csv()`: the bundle's path, else fresh's copy with a message +- [ ] `config_hash` (`R/lnk_log.R` ~L73/117): hash fresh's fallback as `fresh:parameters_habitat_method.csv` when the bundle declares none +- [ ] Add `inst/extdata/configs/dictionary_parameters_habitat_method.csv` and its shape/coverage/no-absent-column tests in `test-dictionaries.R`, following the #282 block +- [ ] `lnk_config_verify` / audit pass clean on all bundles + +## Phase 3: thread through classify + `mad_m3s` on working streams +- [ ] `.lnk_pipeline_prep_network()`: add `fresh::frs_col_join(..., from = "whse_basemapping.fwa_stream_networks_discharge", cols = "mad_m3s", by = "linear_feature_id")` next to the channel_width join. Check that `frs_break_apply` carries it through splits +- [ ] `lnk_pipeline_classify()`: add a `method_csv = NULL` argument, mirroring `thresholds_csv`. Resolve it as `method_csv %||% .lnk_habitat_method_csv(cfg)`, read it, and pass `params_method =` to `frs_habitat_classify()`. Update the roxygen +- [ ] Tests in `test-lnk_pipeline_classify.R`, following the #282 capture pattern with mocked `frs_habitat_classify`: the bundle's table is passed, the undeclared fallback uses fresh's copy and says so, and an explicit `method_csv` wins +- [ ] `lnk_pipeline_connect` stays as it is. `.frs_run_connectivity` is not method-aware (clustering is on gradient), so record that in findings rather than change it + +## Phase 4: live verification (local docker fwapg, scratch schema) +- [ ] State the run decisions (config, scratch schema, WSGs, species) before launching +- [ ] **No-change proof.** Run ADMS with the `default` bundle at HEAD and on the branch, reclassifying on one prepared schema. `streams_habitat` must be byte-identical (digest). The all-`cw` table plus the extra `mad_m3s` column must move nothing +- [ ] Repeat the no-change proof on a second, larger WSG (HORS or BULK) +- [ ] **mad takes effect.** Run a thin bundle (`method_csv`) that sets one WSG with discharge coverage to `mad`. It must run, CO/CH/ST habitat must differ from cw, and BT must get no stream habitat from inheriting rules (fresh's documented behaviour). Record km by species in findings +- [ ] A `mad` WSG with no discharge coverage (BULK has none): confirm and record what happens. Every segment fails the mad rule, so expect zero stream habitat. Surface this, and do not add a workaround + +## Phase 5: docs + follow-ups +- [ ] RUNBOOK §7 "Where habitat thresholds live": add the method table, the fallback, and that `mad_m3s` exists only in the working schema +- [ ] NEWS entry (the version bump comes at merge, through `/gh-pr-merge`) +- [ ] Draft the follow-up issue body in findings, for review and not filed: `lnk_habitat_validate()`'s width relaxation (`R/lnk_habitat_validate.R:759-781`) is channel-width only and gives wrong miss reasons for `mad` groups. `frs_habitat_predicates(model=)` exists to fix it + +## Validation + +- [ ] Tests pass +- [ ] `/code-check` clean on each commit +- [ ] PWF checkboxes match landed work +- [ ] `/planning-archive` on completion From bd2f8fff5696d5cb276091ec063988bceff6058c Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:00:29 -0700 Subject: [PATCH 2/9] Pin fresh v0.36.2 and assert params_method in preflight (#286) fresh 0.35.0 added params_method to frs_habitat_classify(); the floor moves there and Remotes to the latest tag. lnk_preflight_fresh() gains required_formals so an install that exports the function but predates the argument fails before a WSG reaches classify. fresh 0.36.0 reversed frs_db_conn()'s env precedence (PG* before PG_*_SHARE). The vignette context fetch moves to lnk_db_conn(), which still reads PG_*_SHARE first, and stale v0.33.0 / frs_db_conn() notes are corrected. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- CLAUDE.md | 2 +- DESCRIPTION | 4 +- R/lnk_db_conn.R | 9 ++-- R/lnk_habitat_validate.R | 5 +- R/lnk_preflight_fresh.R | 62 ++++++++++++++++++++-- data-raw/wsg_vignette_data.R | 11 ++-- man/lnk_db_conn.Rd | 9 ++-- man/lnk_preflight_fresh.Rd | 10 +++- planning/active/findings.md | 16 ++++++ planning/active/progress.md | 1 + planning/active/review-p1-round1.md | 55 +++++++++++++++++++ planning/active/task_plan.md | 6 +-- tests/testthat/test-lnk_habitat_validate.R | 7 +-- tests/testthat/test-lnk_preflight_fresh.R | 36 +++++++++++++ 14 files changed, 204 insertions(+), 29 deletions(-) create mode 100644 planning/active/review-p1-round1.md diff --git a/CLAUDE.md b/CLAUDE.md index 5b7e320d..488b78ef 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -373,7 +373,7 @@ link is connectivity-system agnostic. Column names are configurable parameters w ## Database Connection -Uses `PG_*_SHARE` env vars (Docker fwapg, same as `frs_db_conn()`) with fallback to standard `PG*` vars. DB is needed for match/score/habitat functions that operate via SQL. The override loading and validation can work with any PostgreSQL. +Uses `PG_*_SHARE` env vars (Docker fwapg) with fallback to standard `PG*` vars. From fresh 0.36.0 `frs_db_conn()` reads them the other way round (`PG*` first), so on a machine that sets both the two connect to different databases (#286). DB is needed for match/score/habitat functions that operate via SQL. The override loading and validation can work with any PostgreSQL. ```r conn <- lnk_db_conn() # reads PG_DB_SHARE, PG_HOST_SHARE, etc. diff --git a/DESCRIPTION b/DESCRIPTION index 9ac567a7..32056821 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -21,7 +21,7 @@ Imports: crate (>= 0.0.2), DBI, digest, - fresh (>= 0.33.0), + fresh (>= 0.35.0), httr, jsonlite, RPostgres, @@ -32,7 +32,7 @@ Imports: yaml Remotes: NewGraphEnvironment/crate, - NewGraphEnvironment/fresh@v0.33.0, + NewGraphEnvironment/fresh@v0.36.2, NewGraphEnvironment/gq Suggests: bcdata, diff --git a/R/lnk_db_conn.R b/R/lnk_db_conn.R index 62f3c63c..1c0c7a55 100644 --- a/R/lnk_db_conn.R +++ b/R/lnk_db_conn.R @@ -14,10 +14,11 @@ #' @return A [DBI::DBIConnection-class] object. #' #' @details -#' Checks `PG_*_SHARE` first (the Docker fwapg convention shared with -#' [fresh::frs_db_conn()]), then standard PostgreSQL variables (`PGHOST`, -#' etc.). This means `lnk_db_conn()` works identically to `frs_db_conn()` -#' when both packages connect to the same database. +#' Checks `PG_*_SHARE` first, then standard PostgreSQL variables +#' (`PGHOST`, etc.). This is the reverse of [fresh::frs_db_conn()] from +#' fresh 0.36.0, which reads `PG*` first and `PG_*_SHARE` only when none +#' of `PG*` is set. On a machine that sets both groups to different +#' targets, the two functions connect to different databases. #' #' @examples #' \dontrun{ diff --git a/R/lnk_habitat_validate.R b/R/lnk_habitat_validate.R index ca024916..3977d9a9 100644 --- a/R/lnk_habitat_validate.R +++ b/R/lnk_habitat_validate.R @@ -818,8 +818,9 @@ lnk_habitat_validate <- function(conn, aoi, cfg, loaded, species, schema, res <- lapply(species, function(sp) { if (!any(seg$species_code == sp)) return(NULL) spp <- .lnk_hv_sp_params(params, loaded$parameters_fresh, sp) - # Channel-width model: the default, and the only one fresh@v0.33.0 (the - # pinned minimum) has; it takes no `model` argument. + # Channel-width model only. fresh >= 0.35.0 takes `model = "mad"`, but the + # miss-reason relaxation below rewrites s.channel_width, so a bundle that + # puts a group on mad is scored here as if it were cw (#286 follow-up). pr <- fresh::frs_habitat_predicates(spp) mins <- .lnk_hv_stage_min(spp) stage_pred <- list( diff --git a/R/lnk_preflight_fresh.R b/R/lnk_preflight_fresh.R index ad9196bc..ffb7ffbe 100644 --- a/R/lnk_preflight_fresh.R +++ b/R/lnk_preflight_fresh.R @@ -25,6 +25,11 @@ #' run without. Defaults to the curated list in `.lnk_fresh_required()`. #' @param required_internal Character vector of non-exported `fresh` #' objects reached via [utils::getFromNamespace()]. +#' @param required_formals Named list mapping a `fresh` function to the +#' arguments link passes it that older releases lack. A symbol can be +#' exported and still reject the call: `frs_habitat_classify()` existed +#' long before it took `params_method`. Defaults to +#' `.lnk_fresh_required_formals()`. #' @param min_version Minimum acceptable `fresh` version. Defaults to the #' floor declared in link's own `DESCRIPTION`, so the pin lives in one #' place. @@ -32,7 +37,8 @@ #' point on a cypher, where the log is all the operator gets. #' #' @return Invisibly, a list with `ok`, `version`, `version_ok`, -#' `missing`, `missing_internal` and `message`. +#' `missing`, `missing_internal`, `missing_formals` (`"fn(arg)"` +#' strings) and `message`. #' #' @family preflight #' @@ -48,11 +54,15 @@ #' bad$missing lnk_preflight_fresh <- function(required = .lnk_fresh_required(), required_internal = .lnk_fresh_required_internal(), + required_formals = .lnk_fresh_required_formals(), min_version = .lnk_fresh_floor(), quiet = FALSE) { stopifnot( is.character(required), length(required) >= 1L, all(nzchar(required)), is.character(required_internal), all(nzchar(required_internal)), + is.list(required_formals), + length(required_formals) == 0L || !is.null(names(required_formals)), + all(nzchar(names(required_formals))), is.character(min_version), length(min_version) == 1L, nzchar(min_version), is.logical(quiet), length(quiet) == 1L, !is.na(quiet)) @@ -62,20 +72,25 @@ lnk_preflight_fresh <- function(required = .lnk_fresh_required(), if (is.null(ns)) { out <- list(ok = FALSE, version = NA_character_, version_ok = FALSE, missing = required, missing_internal = required_internal, + missing_formals = .lnk_fresh_formals_label(required_formals), message = "fresh is not installed or its namespace will not load") } else { missing <- setdiff(required, getNamespaceExports(ns)) missing_internal <- required_internal[ !vapply(required_internal, exists, logical(1), envir = ns, inherits = FALSE)] + missing_formals <- .lnk_fresh_missing_formals(ns, required_formals) version_ok <- !is.na(version) && utils::compareVersion(version, min_version) >= 0L out <- list( - ok = length(missing) == 0L && length(missing_internal) == 0L && version_ok, + ok = length(missing) == 0L && length(missing_internal) == 0L && + length(missing_formals) == 0L && version_ok, version = version, version_ok = version_ok, missing = missing, missing_internal = missing_internal, + missing_formals = missing_formals, message = .lnk_fresh_message(version, min_version, missing, - missing_internal, version_ok)) + missing_internal, version_ok, + missing_formals)) } if (!quiet) message(out$message) @@ -103,6 +118,37 @@ lnk_preflight_fresh <- function(required = .lnk_fresh_required(), "frs_params", "frs_wsg_drainage", "frs_wsg_outlets") } +# Arguments link passes that older fresh releases do not accept. The call +# would fail with "unused argument" only once a WSG reached that phase. +# R/lnk_pipeline_classify.R: params_method arrived in fresh 0.35.0 (#286). +.lnk_fresh_required_formals <- function() { + list(frs_habitat_classify = "params_method") +} + +# "fn(arg)" for each required formal the namespace does not provide. A +# function that is itself absent reports every argument it was asked for; +# the missing export is reported separately. +.lnk_fresh_missing_formals <- function(ns, required_formals) { + out <- character(0) + for (fn in names(required_formals)) { + obj <- if (exists(fn, envir = ns, inherits = FALSE)) { + get(fn, envir = ns) + } + have <- if (is.function(obj)) names(formals(obj)) else character(0) + gone <- setdiff(required_formals[[fn]], have) + if (length(gone)) out <- c(out, sprintf("%s(%s)", fn, gone)) + } + out +} + +.lnk_fresh_formals_label <- function(required_formals) { + out <- character(0) + for (fn in names(required_formals)) { + out <- c(out, sprintf("%s(%s)", fn, required_formals[[fn]])) + } + out +} + # Non-exported fresh objects link reaches via getFromNamespace(). # R/lnk_pipeline_connect.R:101. .lnk_fresh_required_internal <- function() { @@ -166,10 +212,12 @@ lnk_preflight_fresh <- function(required = .lnk_fresh_required(), } .lnk_fresh_message <- function(version, min_version, missing, - missing_internal, version_ok) { + missing_internal, version_ok, + missing_formals = character(0)) { head <- sprintf("fresh %s (floor %s)", if (is.na(version)) "NOT INSTALLED" else version, min_version) - if (length(missing) == 0L && length(missing_internal) == 0L && version_ok) { + if (length(missing) == 0L && length(missing_internal) == 0L && + length(missing_formals) == 0L && version_ok) { return(paste0("[preflight] ", head, " - OK, all required symbols present")) } parts <- character(0) @@ -184,6 +232,10 @@ lnk_preflight_fresh <- function(required = .lnk_fresh_required(), parts <- c(parts, sprintf(" missing internals: %s", paste(missing_internal, collapse = ", "))) } + if (length(missing_formals)) { + parts <- c(parts, sprintf(" missing arguments: %s", + paste(missing_formals, collapse = ", "))) + } parts <- c(parts, " fix: pak::pkg_install(\"NewGraphEnvironment/fresh@\") at or above the floor") paste(c(paste0("[preflight] ", head, " - FAILED"), parts), collapse = "\n") diff --git a/data-raw/wsg_vignette_data.R b/data-raw/wsg_vignette_data.R index 222ba4bb..5dc30516 100644 --- a/data-raw/wsg_vignette_data.R +++ b/data-raw/wsg_vignette_data.R @@ -28,7 +28,7 @@ # `;DAM` tokens are correct. READ it; do not recompute (a standalone # single-WSG re-run would diverge on those segments). # * CONTEXT BASEMAPPING — the db_newgraph full catalog via -# fresh::frs_db_conn() (localhost:63333, dbname `bcfishpass`). reserves +# lnk_db_conn() (PG_*_SHARE: the 63333 tunnel, dbname `bcfishpass`). reserves # (whse_admin_boundaries), parks (whse_tantalis), roads + railways # (whse_basemapping.transport_line / gba_railway_tracks_sp) are not in # the FWA-only local subset. These are the same `fetch_layer` queries @@ -195,16 +195,19 @@ named_streams <- sf::st_zm(named_streams, drop = TRUE) # --- context basemapping from the db_newgraph full catalog (63333) ------------ # reserves / parks / roads / railways are absent from the FWA-only local -# subset. Pull them the way flooded + the Peace report do: frs_db_conn(). +# subset. Pull them the way flooded + the Peace report do. lnk_db_conn(), not +# fresh::frs_db_conn(): from fresh 0.36.0 the latter reads PG* before +# PG_*_SHARE, which on a machine with both set is the local fwapg, not the +# tunnel this needs (#286). boundary_wkt <- sf::st_as_text(sf::st_union(sf::st_geometry(boundary))) intersect_clause <- function(geom_col = "geom") { sprintf("ST_Intersects(%s, ST_GeomFromText('%s', 3005))", geom_col, boundary_wkt) } -conn_ctx <- try(fresh::frs_db_conn(), silent = TRUE) +conn_ctx <- try(lnk_db_conn(), silent = TRUE) context_layers <- list() if (inherits(conn_ctx, "try-error") || is.null(conn_ctx)) { - message("[wsg_vignette_data] frs_db_conn() unavailable — context layers ", + message("[wsg_vignette_data] lnk_db_conn() unavailable — context layers ", "(reserves/parks/roads/railways) skipped. Bring up the 63333 ", "db_newgraph tunnel to ship them (soul/skills/db-newgraph).") } else { diff --git a/man/lnk_db_conn.Rd b/man/lnk_db_conn.Rd index 58712cd5..20a28be4 100644 --- a/man/lnk_db_conn.Rd +++ b/man/lnk_db_conn.Rd @@ -33,10 +33,11 @@ by fresh and fpr (Docker-hosted fwapg). Falls back to standard \verb{PG*} variables for local PostgreSQL. } \details{ -Checks \verb{PG_*_SHARE} first (the Docker fwapg convention shared with -\code{\link[fresh:frs_db_conn]{fresh::frs_db_conn()}}), then standard PostgreSQL variables (\code{PGHOST}, -etc.). This means \code{lnk_db_conn()} works identically to \code{frs_db_conn()} -when both packages connect to the same database. +Checks \verb{PG_*_SHARE} first, then standard PostgreSQL variables +(\code{PGHOST}, etc.). This is the reverse of \code{\link[fresh:frs_db_conn]{fresh::frs_db_conn()}} from +fresh 0.36.0, which reads \verb{PG*} first and \verb{PG_*_SHARE} only when none +of \verb{PG*} is set. On a machine that sets both groups to different +targets, the two functions connect to different databases. } \examples{ \dontrun{ diff --git a/man/lnk_preflight_fresh.Rd b/man/lnk_preflight_fresh.Rd index 69793140..149d4dac 100644 --- a/man/lnk_preflight_fresh.Rd +++ b/man/lnk_preflight_fresh.Rd @@ -7,6 +7,7 @@ lnk_preflight_fresh( required = .lnk_fresh_required(), required_internal = .lnk_fresh_required_internal(), + required_formals = .lnk_fresh_required_formals(), min_version = .lnk_fresh_floor(), quiet = FALSE ) @@ -18,6 +19,12 @@ run without. Defaults to the curated list in \code{.lnk_fresh_required()}.} \item{required_internal}{Character vector of non-exported \code{fresh} objects reached via \code{\link[utils:getFromNamespace]{utils::getFromNamespace()}}.} +\item{required_formals}{Named list mapping a \code{fresh} function to the +arguments link passes it that older releases lack. A symbol can be +exported and still reject the call: \code{frs_habitat_classify()} existed +long before it took \code{params_method}. Defaults to +\code{.lnk_fresh_required_formals()}.} + \item{min_version}{Minimum acceptable \code{fresh} version. Defaults to the floor declared in link's own \code{DESCRIPTION}, so the pin lives in one place.} @@ -27,7 +34,8 @@ point on a cypher, where the log is all the operator gets.} } \value{ Invisibly, a list with \code{ok}, \code{version}, \code{version_ok}, -\code{missing}, \code{missing_internal} and \code{message}. +\code{missing}, \code{missing_internal}, \code{missing_formals} (\code{"fn(arg)"} +strings) and \code{message}. } \description{ \code{link} calls \verb{fresh::} in a dozen places with no \code{requireNamespace()} diff --git a/planning/active/findings.md b/planning/active/findings.md index 8b7fcf09..69f47389 100644 --- a/planning/active/findings.md +++ b/planning/active/findings.md @@ -28,6 +28,22 @@ glacier-fed basins. That is measurable: compare `cw` and `mad` classification wh Thresholds in both currencies, with verbatim quotes, are in NewGraphEnvironment/knowledge#30 (`research/ip_models.md`). +## Phase 1 — fresh pin (2026-10-01) + +- Installed fresh before this branch read `0.34.0`, a dev install, yet already had + `params_method`. That is a version string that is not the release: the reason the + preflight guard asserts the formal, not the version. Now `fresh 0.36.2` (`github`, + `v0.36.2`, `e3a37f0`). +- `lnk_preflight_fresh()` gains `required_formals`, default + `list(frs_habitat_classify = "params_method")`. Stubbing + `.lnk_fresh_missing_formals` to return nothing turns 3 tests red, so the guard fires. +- **fresh 0.36.0 `frs_db_conn()` order flip does bite here.** Inside R on this machine + both `PG*` and `PG_*_SHARE` are set, with the same host but a different port and + database. So a bare `fresh::frs_db_conn()` in `data-raw/wsg_vignette_data.R:204` would + have moved from the tunnel to the local fwapg. It is swapped to `lnk_db_conn()`, which + still reads `PG_*_SHARE` first. `lnk_db_conn()`'s roxygen claimed to work "identically + to `frs_db_conn()`", which is now false, so that is corrected too. + ## Errors Encountered | Error | Resolution | diff --git a/planning/active/progress.md b/planning/active/progress.md index 6b13b726..d0d332df 100644 --- a/planning/active/progress.md +++ b/planning/active/progress.md @@ -6,3 +6,4 @@ - Created branch `286-thread-fresh-params-method-per-wsg-cw-ma` off main - Scaffolded PWF baseline from issue #286 with approved phases - Next: start Phase 1 +- Phase 1: fresh pin -> v0.36.2 (floor 0.35.0); `lnk_preflight_fresh(required_formals=)` asserts `frs_habitat_classify(params_method)`; vignette script off `frs_db_conn()` (fresh 0.36.0 precedence flip). Code-check round 1 clean; it flagged stale text (validator/test comments citing v0.33.0, CLAUDE.md `frs_db_conn` line, a skip message), fixed in the same commit diff --git a/planning/active/review-p1-round1.md b/planning/active/review-p1-round1.md new file mode 100644 index 00000000..ff3c6d14 --- /dev/null +++ b/planning/active/review-p1-round1.md @@ -0,0 +1,55 @@ +# Review — #286 Phase 1, round 1 + +Diff: DESCRIPTION (fresh floor 0.35.0, Remotes v0.36.2), R/lnk_db_conn.R (roxygen), +R/lnk_preflight_fresh.R (`required_formals`), data-raw/wsg_vignette_data.R +(`fresh::frs_db_conn()` -> `lnk_db_conn()`), tests/testthat/test-lnk_preflight_fresh.R. + +## Clean + +No issues found that could cause failures, security problems or data loss. + +### What was checked + +- `test-lnk_preflight_fresh.R` under `NOT_CRAN=true` + `load_all`: FAIL 0 / PASS 38. + `lnk_preflight_fresh()` against installed fresh 0.36.2: ok, `missing_formals` = `chr(0)`. +- Installed `fresh::frs_habitat_classify` formals include `params_method` (default NULL -> + bundled all-`cw` table), so link's current calls (which do not pass it yet) are unchanged. +- Validation edge cases: `required_formals = list()` passes (names NULL -> `nzchar(NULL)` is + `logical(0)`, `all()` TRUE); an unnamed list is rejected; a partially named list is rejected + by `nzchar("")`. `[[fn]]` is exact-match (no `$` partial match). `sprintf()` over an empty + arg vector yields `character(0)`, so no phantom entry. `.lnk_fresh_message` gains a trailing + defaulted arg; its only caller passes it. +- fresh v0.33.0..v0.36.2 R/ changes (`git diff --stat -- R`): frs_db_conn, frs_habitat, + frs_habitat_classify, frs_habitat_predicates, frs_network_segment, frs_params, utils. + - `frs_db_conn()`: no other live call site in link R/, data-raw/ or tests/ (only the one + swapped here). `lnk_db_conn()` itself does not call it. + - `frs_habitat_predicates(sp_params)` (lnk_habitat_validate.R:823): `model` defaults to + `"cw"`; the cw SQL text is the same shape (`s.channel_width >= x AND s.channel_width <= y`, + `s.gradient ...`), so `.lnk_hv_relax()`'s `\bs\.channel_width\b` / `\bs\.gradient\b` + substitutions still match. + - `frs_params(csv =, rules_yaml =)` (classify:91, connect:94, validate:803): no conn path, + so the 0.36.0 env-var flip does not reach it. `mad` rule key now accepted instead of + erroring — no link rules use it. + - `.frs_sql_num()` now `unlist()`s and renders `Inf` as `'Infinity'::double precision`; + finite numerics render identically. + - `.frs_run_connectivity` (reached via getFromNamespace) is unchanged across the range. +- Remotes collision (code-check-r "Two repos pinning the same remote at different tags"): + no other local NGE DESCRIPTION pins `fresh@`; breaks and flooded take it unpinned; + crate and gq do not depend on fresh. +- `wsg_vignette_data.R`: on a machine without `PG_*_SHARE`, `lnk_db_conn()` now falls back to + `PG*` (and connects) where old `frs_db_conn()` would have stopped; each `fetch_ctx()` is + wrapped in `try()` and skips on error, so the script degrades rather than fails. + +### Non-blocking notes (stale text, not defects) + +- data-raw/wsg_vignette_data.R:210 — the skip message still says `frs_db_conn() unavailable`; + the call is now `lnk_db_conn()`. Misleading to an operator reading the log. +- R/lnk_habitat_validate.R:819-821 and tests/testthat/test-lnk_habitat_validate.R:66-67 — + comments say "fresh@v0.33.0 (the pinned minimum)". The floor is now 0.35.0, whose + `frs_habitat_predicates()` does take `model`. The test still passes (it asserts a one-arg + call), but its stated rationale is now false. +- CLAUDE.md:376 — "`PG_*_SHARE` env vars (Docker fwapg, same as `frs_db_conn()`)" is no longer + true from fresh 0.36.0; the roxygen was corrected but this sentence was not. +- `.lnk_fresh_required_formals()` comment cites R/lnk_pipeline_classify.R as passing + `params_method`; it does not yet (Phase 2/3). Expected per the plan, but the comment reads + as current fact. diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md index 9918800e..853060f6 100644 --- a/planning/active/task_plan.md +++ b/planning/active/task_plan.md @@ -31,9 +31,9 @@ needs adding there. 3. Provenance comes through `config_hash` only. No new log table and no new log column. ## Phase 1: fresh pin + capability guard -- [ ] DESCRIPTION: `fresh (>= 0.35.0)`, `Remotes: NewGraphEnvironment/fresh@v0.36.2`. Reinstall fresh at the tag and confirm `params_method` is in `formals(fresh::frs_habitat_classify)` -- [ ] `lnk_preflight_fresh()` (`R/lnk_preflight_fresh.R`): assert the `params_method` formal on `frs_habitat_classify` (a capability, not a version). Add a test that restores the defect and shows the guard fires -- [ ] Check `data-raw/wsg_vignette_data.R`'s bare `fresh::frs_db_conn()` against the 0.36.0 behaviour change, and note the result in findings +- [x] DESCRIPTION: `fresh (>= 0.35.0)`, `Remotes: NewGraphEnvironment/fresh@v0.36.2`. Reinstall fresh at the tag and confirm `params_method` is in `formals(fresh::frs_habitat_classify)` +- [x] `lnk_preflight_fresh()` (`R/lnk_preflight_fresh.R`): assert the `params_method` formal on `frs_habitat_classify` (a capability, not a version). Add a test that restores the defect and shows the guard fires +- [x] Check `data-raw/wsg_vignette_data.R`'s bare `fresh::frs_db_conn()` against the 0.36.0 behaviour change, and note the result in findings ## Phase 2: method table as bundle data - [ ] Add `parameters_habitat_method.csv` to `bcfishpass`, `default`, `default_extrabreaks` and `default_rearbreaks`: a frozen copy of `smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv` (188 groups, all `cw`). Diff it against fresh's bundled copy first and record the result. `default_tuned` inherits it through `extends` diff --git a/tests/testthat/test-lnk_habitat_validate.R b/tests/testthat/test-lnk_habitat_validate.R index 358b2d75..d51ff259 100644 --- a/tests/testthat/test-lnk_habitat_validate.R +++ b/tests/testthat/test-lnk_habitat_validate.R @@ -62,9 +62,10 @@ test_that(".lnk_hv_stage_min takes the spawn gradient floor from parameters_fres expect_identical(m$rear[["width"]], 1.5) }) -test_that("the predicate call passes no argument fresh@v0.33.0 lacks", { - # DESCRIPTION pins fresh >= 0.33.0, whose frs_habitat_predicates() takes - # only `sp_params`; `model =` arrived later and would error there. +test_that("the predicate call stays on the channel-width model", { + # The validator relaxes s.channel_width, so it is cw-only. fresh >= 0.35.0 + # accepts `model =`; passing it here needs the relaxation reworked first + # (#286 follow-up), and this pins that it has not happened by accident. calls <- list() walk <- function(e) { if (is.call(e)) { diff --git a/tests/testthat/test-lnk_preflight_fresh.R b/tests/testthat/test-lnk_preflight_fresh.R index fa979e39..0f8731ad 100644 --- a/tests/testthat/test-lnk_preflight_fresh.R +++ b/tests/testthat/test-lnk_preflight_fresh.R @@ -4,6 +4,7 @@ test_that("lnk_preflight_fresh passes against the real required set", { expect_true(res$ok) expect_length(res$missing, 0L) expect_length(res$missing_internal, 0L) + expect_length(res$missing_formals, 0L) expect_true(res$version_ok) }) @@ -30,6 +31,41 @@ test_that("lnk_preflight_fresh fails when a required internal is absent", { expect_identical(res$missing_internal, ".frs_not_an_internal") }) +test_that("lnk_preflight_fresh fails when a required argument is absent (#286)", { + skip_if_not_installed("fresh") + absent <- "not_a_formal_of_frs_habitat_classify" + expect_false(absent %in% names(formals(fresh::frs_habitat_classify))) + res <- lnk_preflight_fresh( + required_formals = list(frs_habitat_classify = c("params_method", absent)), + quiet = TRUE) + expect_false(res$ok) + expect_identical(res$missing_formals, + sprintf("frs_habitat_classify(%s)", absent)) + expect_match(res$message, "missing arguments", fixed = TRUE) +}) + +test_that("a pre-0.35.0 frs_habitat_classify signature is caught (#286)", { + # The defect itself: a fresh that exports frs_habitat_classify without + # params_method. Built as a fake namespace so it needs no old install. + ns <- new.env() + ns$frs_habitat_classify <- function(conn, table, to, species, params, + params_fresh, gate, label_block, + barrier_overrides, overwrite, + verbose) NULL + expect_identical( + .lnk_fresh_missing_formals(ns, .lnk_fresh_required_formals()), + "frs_habitat_classify(params_method)") + # And a function that is absent altogether reports its arguments too. + expect_identical( + .lnk_fresh_missing_formals(new.env(), .lnk_fresh_required_formals()), + "frs_habitat_classify(params_method)") +}) + +test_that("required formals name functions in the required export set", { + expect_true(all(names(.lnk_fresh_required_formals()) %in% + .lnk_fresh_required())) +}) + test_that("lnk_preflight_fresh fails a version floor above the installed", { skip_if_not_installed("fresh") res <- lnk_preflight_fresh(min_version = "999.0.0", quiet = TRUE) From 6672168c2bbcf506151d44f7678fa7524c4531d0 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:19:56 -0700 Subject: [PATCH 3/9] Ship a frozen cw/mad method table in every bundle (#286) Each base bundle declares parameters_habitat_method.csv, a frozen copy of bcfishpass example_newgraph (188 groups, all cw). Its provenance source is deliberately not the bcfishpass URL: csv-sync auto-merges any entry with that source, which would let an upstream cw -> mad flip change default unreviewed. A bundle that declares none falls back to fresh's copy, hashed into config_hash by a fixed name. The discharge table joins log_input's primitives, and the thresholds dictionary no longer says *_mad_* never apply. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- R/lnk_config.R | 18 ++ R/lnk_log.R | 19 +- R/lnk_preflight_vintage.R | 2 +- inst/extdata/configs/bcfishpass/README.md | 1 + inst/extdata/configs/bcfishpass/config.yaml | 9 + .../bcfishpass/parameters_habitat_method.csv | 189 ++++++++++++++++++ inst/extdata/configs/default/README.md | 1 + inst/extdata/configs/default/config.yaml | 9 + .../default/parameters_habitat_method.csv | 189 ++++++++++++++++++ .../configs/default_extrabreaks/README.md | 1 + .../configs/default_extrabreaks/config.yaml | 9 + .../parameters_habitat_method.csv | 189 ++++++++++++++++++ .../configs/default_rearbreaks/README.md | 1 + .../configs/default_rearbreaks/config.yaml | 9 + .../parameters_habitat_method.csv | 189 ++++++++++++++++++ inst/extdata/configs/default_tuned/README.md | 1 + .../dictionary_parameters_habitat_method.csv | 3 + ...ctionary_parameters_habitat_thresholds.csv | 8 +- man/lnk_preflight_vintage.Rd | 2 +- planning/active/findings.md | 77 +++++++ planning/active/progress.md | 1 + planning/active/review-p23-round2.md | 102 ++++++++++ planning/active/review-plan.md | 25 +++ planning/active/review-round3.md | 101 ++++++++++ planning/active/task_plan.md | 12 +- tests/testthat/test-dictionaries.R | 53 +++++ tests/testthat/test-lnk_config.R | 25 +++ tests/testthat/test-lnk_log.R | 32 +++ tests/testthat/test-lnk_preflight_vintage.R | 2 +- 29 files changed, 1265 insertions(+), 14 deletions(-) create mode 100644 inst/extdata/configs/bcfishpass/parameters_habitat_method.csv create mode 100644 inst/extdata/configs/default/parameters_habitat_method.csv create mode 100644 inst/extdata/configs/default_extrabreaks/parameters_habitat_method.csv create mode 100644 inst/extdata/configs/default_rearbreaks/parameters_habitat_method.csv create mode 100644 inst/extdata/configs/dictionary_parameters_habitat_method.csv create mode 100644 planning/active/review-p23-round2.md create mode 100644 planning/active/review-plan.md create mode 100644 planning/active/review-round3.md diff --git a/R/lnk_config.R b/R/lnk_config.R index c28d04e9..1350a5ee 100644 --- a/R/lnk_config.R +++ b/R/lnk_config.R @@ -338,6 +338,24 @@ print.lnk_config <- function(x, ...) { fresh_path } +# Path to the per-watershed-group habitat model table (`cw` or `mad`) a +# config runs with (#286). Same contract as the thresholds CSV above: the +# bundle's own `files: parameters_habitat_method:`, else fresh's shipped copy +# (all `cw`), with a message because that copy sits outside the config's +# provenance. +.lnk_habitat_method_csv <- function(cfg) { + path <- cfg$files$parameters_habitat_method$path + if (!is.null(path)) { + return(path) + } + fresh_path <- system.file("extdata", "parameters_habitat_method.csv", + package = "fresh") + message("config '", cfg$name %||% "", "' declares no ", + "files$parameters_habitat_method; using fresh's copy: ", + fresh_path) + fresh_path +} + # Absolute path of a provenance entry. Keys are relative to the bundle that # declared them: the leaf for its own entries, `.dir` for inherited ones. .lnk_provenance_path <- function(cfg, rel) { diff --git a/R/lnk_log.R b/R/lnk_log.R index e4dc4b76..68a95c63 100644 --- a/R/lnk_log.R +++ b/R/lnk_log.R @@ -73,6 +73,10 @@ fallback <- if (is.null(cfg$files$parameters_habitat_thresholds)) { suppressMessages(.lnk_habitat_thresholds_csv(cfg)) } + # Same for the cw/mad method table (#286). + fallback_method <- if (is.null(cfg$files$parameters_habitat_method)) { + suppressMessages(.lnk_habitat_method_csv(cfg)) + } # Name each file relative to the bundle that holds it, never by absolute # path, so the hash is the same on every host. Leaf files keep their plain @@ -122,6 +126,16 @@ }) } + if (length(fallback_method) == 1L && nzchar(fallback_method)) { + paths <- c(paths, fallback_method) + rel <- c(rel, "fresh:parameters_habitat_method.csv") + digests <- c(digests, if (file.exists(fallback_method)) { + digest::digest(file = fallback_method, algo = "sha256") + } else { + "MISSING" + }) + } + # radix: byte order, not LC_COLLATE, or the hash differs between hosts whose # locales sort `_control.csv` and `.csv` differently. ord <- order(rel, method = "radix") @@ -189,6 +203,9 @@ table_name = c( "whse_basemapping.fwa_stream_networks_sp", "whse_basemapping.fwa_stream_networks_channel_width", + # mean annual discharge: drives classification for groups a bundle's + # parameters_habitat_method puts on `mad` (#286) + "whse_basemapping.fwa_stream_networks_discharge", "whse_basemapping.fwa_stream_networks_order_parent", "whse_basemapping.fwa_lakes_poly", "whse_basemapping.fwa_wetlands_poly", @@ -206,7 +223,7 @@ "fresh.modelled_stream_crossings" ), source = c( - rep(fwapg, 7L), + rep(fwapg, 8L), "bcfishobs", "bcdata bc2pg", "CABD", diff --git a/R/lnk_preflight_vintage.R b/R/lnk_preflight_vintage.R index 05807a16..0942045d 100644 --- a/R/lnk_preflight_vintage.R +++ b/R/lnk_preflight_vintage.R @@ -8,7 +8,7 @@ #' May inputs and three on August inputs, produced one consolidated table #' set, and said nothing about it anywhere (link#246). #' -#' The other seven tables in `.lnk_input_primitives()` are bulk-restored +#' The FWA tables in `.lnk_input_primitives()` are bulk-restored #' FWA. They are never `ANALYZE`d, so they carry no vintage at all and are #' not an axis this can measure — including them would mean every host #' failing forever on data that is not the staleness risk. diff --git a/inst/extdata/configs/bcfishpass/README.md b/inst/extdata/configs/bcfishpass/README.md index 7deed871..80a3d8ef 100644 --- a/inst/extdata/configs/bcfishpass/README.md +++ b/inst/extdata/configs/bcfishpass/README.md @@ -11,6 +11,7 @@ Reproduces bcfishpass output exactly for regression. All five watershed groups ( | `dimensions.csv` | Source of `rules.yaml` — species × habitat biology encoded for bcfishpass-match | | `parameters_fresh.csv` | Per-species fresh overrides (spawn_gradient_min, observation_threshold, etc.) | | `parameters_habitat_thresholds.csv` | Per-species gradient / channel-width / MAD / lake-area thresholds and edge types. Vendored from fresh (= bcfishpass `parameters/example_newgraph` + fresh's edge-type columns); provenance in `config.yaml`. **Frozen parity input** — not touched by the csv-sync workflow, so it changes only when someone changes it on purpose. | +| `parameters_habitat_method.csv` | Per-watershed-group habitat size model, `cw` (channel width) or `mad` (mean annual discharge), handed to fresh by classify. All `cw`; a frozen copy of bcfishpass `parameters/example_newgraph`, not csv-synced, so moving a group to `mad` is a reviewed edit (update its `checksum` in `config.yaml`). Columns in `configs/dictionary_parameters_habitat_method.csv`. | | `overrides/` | Synced from `smnorris/bcfishpass/data/` — expert-curated corrections + confirmed habitat + observation exclusions. Redistributed under `LICENSE-bcfishpass` at the repo root. | The bundle is consumed via `lnk_config("bcfishpass")` (manifest only — paths + provenance) and `lnk_load_overrides(cfg)` (canonical-shape tibbles). `user_habitat_classification` routes through `crate::crt_ingest()` for variant-stable ingest; the rest fall through to local CSV reads. diff --git a/inst/extdata/configs/bcfishpass/config.yaml b/inst/extdata/configs/bcfishpass/config.yaml index 35aee5fe..3cd84778 100644 --- a/inst/extdata/configs/bcfishpass/config.yaml +++ b/inst/extdata/configs/bcfishpass/config.yaml @@ -17,6 +17,8 @@ files: path: parameters_fresh.csv parameters_habitat_thresholds: path: parameters_habitat_thresholds.csv + parameters_habitat_method: + path: parameters_habitat_method.csv user_habitat_classification: source: bcfp path: overrides/user_habitat_classification.csv @@ -132,6 +134,13 @@ provenance: synced: '2026-09-25' checksum: sha256:7b904a18c29c724737ff24a27750bf05d7bfa281aa4fc1f7ac27c00238db042d shape_checksum: sha256:8de9bc1fcc2c703176a57d8a982bd26da1c49a885af3053f00078003a7e9283f + parameters_habitat_method.csv: + source: link (frozen copy; not csv-synced, link#286). Moving a group to mad is a reviewed edit here + derived_from: smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv + upstream_sha: 1fae4eae67284fbe7950dbdb043b589a20b428e2 + synced: '2026-10-01' + checksum: sha256:9802bb71bdd7f7d296e917d138567f096a69dab12dbc7c0f331e55206e5f6a0a + shape_checksum: sha256:cf285e3c204249a6b628424979ef780baa271add44cc8ea6512a44d6671a9eb5 overrides/user_habitat_classification.csv: source: https://github.com/smnorris/bcfishpass path: data/user_habitat_classification.csv diff --git a/inst/extdata/configs/bcfishpass/parameters_habitat_method.csv b/inst/extdata/configs/bcfishpass/parameters_habitat_method.csv new file mode 100644 index 00000000..be6081b3 --- /dev/null +++ b/inst/extdata/configs/bcfishpass/parameters_habitat_method.csv @@ -0,0 +1,189 @@ +watershed_group_code,model +ADMS,cw +ALBN,cw +ATNA,cw +BABL,cw +BABR,cw +BARR,cw +BBAR,cw +BELA,cw +BIGC,cw +BLAR,cw +BONP,cw +BOWR,cw +BRID,cw +BRKS,cw +BULK,cw +BULL,cw +CAMB,cw +CANO,cw +CARP,cw +CARR,cw +CHES,cw +CHIL,cw +CHIR,cw +CHWK,cw +CLAY,cw +CLRH,cw +CLWR,cw +COLR,cw +COMX,cw +COTR,cw +COWN,cw +CRKD,cw +DEAD,cw +DOGC,cw +DRIR,cw +DUNC,cw +ELKR,cw +EUCH,cw +EUCL,cw +FINA,cw +FINL,cw +FIRE,cw +FOXR,cw +FRAN,cw +FRCN,cw +GLAR,cw +GOLD,cw +GRAI,cw +GRNL,cw +GUIC,cw +HARR,cw +HERR,cw +HOLB,cw +HOMA,cw +HORS,cw +INGR,cw +INKR,cw +JENR,cw +JERV,cw +KEEC,cw +KETL,cw +KHOR,cw +KHTZ,cw +KINR,cw +KISP,cw +KITL,cw +KITR,cw +KLIN,cw +KLUM,cw +KNIG,cw +KOTL,cw +KOTR,cw +KSHR,cw +KTSU,cw +KUMR,cw +LARL,cw +LBIR,cw +LCHL,cw +LCHR,cw +LDEN,cw +LEUT,cw +LFRA,cw +LILL,cw +LISR,cw +LKEL,cw +LNAR,cw +LNIC,cw +LNRS,cw +LNTH,cw +LOMI,cw +LRDO,cw +LSAL,cw +LSKE,cw +LSTR,cw +LTRE,cw +MAHD,cw +MBNK,cw +MCGR,cw +MESC,cw +MESI,cw +MFRA,cw +MIDR,cw +MORI,cw +MORK,cw +MORR,cw +MSKE,cw +MSTR,cw +MURT,cw +MUSK,cw +NAHR,cw +NAKR,cw +NARC,cw +NASC,cw +NASR,cw +NATR,cw +NAZR,cw +NBNK,cw +NECL,cw +NECR,cw +NEVI,cw +NICL,cw +NIEL,cw +NIMP,cw +OKAN,cw +OSPK,cw +OWIK,cw +PARA,cw +PARK,cw +PARS,cw +PCEA,cw +PINE,cw +PORI,cw +QUES,cw +REVL,cw +SAJR,cw +SALM,cw +SALR,cw +SANJ,cw +SETN,cw +SEYM,cw +SHER,cw +SHUL,cw +SIML,cw +SKGT,cw +SLOC,cw +SMAR,cw +SQAM,cw +STHM,cw +STIR,cw +STUL,cw +STUR,cw +SUST,cw +SWIR,cw +TABR,cw +TAHR,cw +TAHS,cw +TAKL,cw +TASR,cw +TATR,cw +TAYR,cw +TESR,cw +THOM,cw +TOBA,cw +TOOD,cw +TSAY,cw +TSIT,cw +TWAC,cw +UARL,cw +UBIR,cw +UCHR,cw +UDEN,cw +UEUT,cw +UFRA,cw +UJER,cw +UNAR,cw +UNRS,cw +UNTH,cw +UNUR,cw +UOMI,cw +UPCE,cw +USHU,cw +USKE,cw +USTK,cw +UTRE,cw +VICT,cw +WILL,cw +WORC,cw +ZYMO,cw diff --git a/inst/extdata/configs/default/README.md b/inst/extdata/configs/default/README.md index f193c39d..2be388cb 100644 --- a/inst/extdata/configs/default/README.md +++ b/inst/extdata/configs/default/README.md @@ -24,6 +24,7 @@ Not in this config: | `dimensions.csv` | Source of `rules.yaml` — species × habitat biology encoded for NewGraph defaults. Source of truth is `inst/extdata/parameters_habitat_dimensions.csv` (copied in here on bundle assembly) | | `parameters_fresh.csv` | Per-species fresh overrides (spawn_gradient_min, observation_threshold, etc.) | | `parameters_habitat_thresholds.csv` | Per-species gradient / channel-width / MAD / lake-area thresholds and edge types, read by classify and connect. Starts identical to fresh's copy; provenance in `config.yaml`. | +| `parameters_habitat_method.csv` | Per-watershed-group habitat size model, `cw` (channel width) or `mad` (mean annual discharge), handed to fresh by classify. All `cw`; a frozen copy of bcfishpass `parameters/example_newgraph`, not csv-synced, so moving a group to `mad` is a reviewed edit (update its `checksum` in `config.yaml`). Columns in `configs/dictionary_parameters_habitat_method.csv`. | | `species_pooling.csv` | Which observation species count as evidence for which model species, and where: one dated, sourced row per decision, scoped to a region, sub-region (`inst/extdata/wsg_regions.csv`) or WSG. Either side can be a group from `species_groups.csv`. Read by `lnk_species_pooling()`; columns in `configs/dictionary_species_pooling.csv`. Anything unlisted is not pooled. After an edit, update its `checksum` in `config.yaml` (`lnk_config_verify()` reports the drift until you do). | | `species_groups.csv` | Named sets of species (`SALMON`, `CHAR`) usable on either side of a pooling row. | | `overrides/` | Shared jurisdiction data — same barrier corrections, PSCIS status overrides, observation exclusions, habitat confirmations as the bcfishpass variant. These are BC-specific facts, not method choices. Redistributed under `LICENSE-bcfishpass` at the repo root. | diff --git a/inst/extdata/configs/default/config.yaml b/inst/extdata/configs/default/config.yaml index e6b32f2e..0aef0db4 100644 --- a/inst/extdata/configs/default/config.yaml +++ b/inst/extdata/configs/default/config.yaml @@ -25,6 +25,8 @@ files: path: parameters_fresh.csv parameters_habitat_thresholds: path: parameters_habitat_thresholds.csv + parameters_habitat_method: + path: parameters_habitat_method.csv species_pooling: path: species_pooling.csv species_groups: @@ -142,6 +144,13 @@ provenance: synced: '2026-09-25' checksum: sha256:7b904a18c29c724737ff24a27750bf05d7bfa281aa4fc1f7ac27c00238db042d shape_checksum: sha256:8de9bc1fcc2c703176a57d8a982bd26da1c49a885af3053f00078003a7e9283f + parameters_habitat_method.csv: + source: link (frozen copy; not csv-synced, link#286). Moving a group to mad is a reviewed edit here + derived_from: smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv + upstream_sha: 1fae4eae67284fbe7950dbdb043b589a20b428e2 + synced: '2026-10-01' + checksum: sha256:9802bb71bdd7f7d296e917d138567f096a69dab12dbc7c0f331e55206e5f6a0a + shape_checksum: sha256:cf285e3c204249a6b628424979ef780baa271add44cc8ea6512a44d6671a9eb5 species_pooling.csv: source: link (hand-authored; link#290). The DV->BT pooling tracker, one row per decision synced: '2026-09-27' diff --git a/inst/extdata/configs/default/parameters_habitat_method.csv b/inst/extdata/configs/default/parameters_habitat_method.csv new file mode 100644 index 00000000..be6081b3 --- /dev/null +++ b/inst/extdata/configs/default/parameters_habitat_method.csv @@ -0,0 +1,189 @@ +watershed_group_code,model +ADMS,cw +ALBN,cw +ATNA,cw +BABL,cw +BABR,cw +BARR,cw +BBAR,cw +BELA,cw +BIGC,cw +BLAR,cw +BONP,cw +BOWR,cw +BRID,cw +BRKS,cw +BULK,cw +BULL,cw +CAMB,cw +CANO,cw +CARP,cw +CARR,cw +CHES,cw +CHIL,cw +CHIR,cw +CHWK,cw +CLAY,cw +CLRH,cw +CLWR,cw +COLR,cw +COMX,cw +COTR,cw +COWN,cw +CRKD,cw +DEAD,cw +DOGC,cw +DRIR,cw +DUNC,cw +ELKR,cw +EUCH,cw +EUCL,cw +FINA,cw +FINL,cw +FIRE,cw +FOXR,cw +FRAN,cw +FRCN,cw +GLAR,cw +GOLD,cw +GRAI,cw +GRNL,cw +GUIC,cw +HARR,cw +HERR,cw +HOLB,cw +HOMA,cw +HORS,cw +INGR,cw +INKR,cw +JENR,cw +JERV,cw +KEEC,cw +KETL,cw +KHOR,cw +KHTZ,cw +KINR,cw +KISP,cw +KITL,cw +KITR,cw +KLIN,cw +KLUM,cw +KNIG,cw +KOTL,cw +KOTR,cw +KSHR,cw +KTSU,cw +KUMR,cw +LARL,cw +LBIR,cw +LCHL,cw +LCHR,cw +LDEN,cw +LEUT,cw +LFRA,cw +LILL,cw +LISR,cw +LKEL,cw +LNAR,cw +LNIC,cw +LNRS,cw +LNTH,cw +LOMI,cw +LRDO,cw +LSAL,cw +LSKE,cw +LSTR,cw +LTRE,cw +MAHD,cw +MBNK,cw +MCGR,cw +MESC,cw +MESI,cw +MFRA,cw +MIDR,cw +MORI,cw +MORK,cw +MORR,cw +MSKE,cw +MSTR,cw +MURT,cw +MUSK,cw +NAHR,cw +NAKR,cw +NARC,cw +NASC,cw +NASR,cw +NATR,cw +NAZR,cw +NBNK,cw +NECL,cw +NECR,cw +NEVI,cw +NICL,cw +NIEL,cw +NIMP,cw +OKAN,cw +OSPK,cw +OWIK,cw +PARA,cw +PARK,cw +PARS,cw +PCEA,cw +PINE,cw +PORI,cw +QUES,cw +REVL,cw +SAJR,cw +SALM,cw +SALR,cw +SANJ,cw +SETN,cw +SEYM,cw +SHER,cw +SHUL,cw +SIML,cw +SKGT,cw +SLOC,cw +SMAR,cw +SQAM,cw +STHM,cw +STIR,cw +STUL,cw +STUR,cw +SUST,cw +SWIR,cw +TABR,cw +TAHR,cw +TAHS,cw +TAKL,cw +TASR,cw +TATR,cw +TAYR,cw +TESR,cw +THOM,cw +TOBA,cw +TOOD,cw +TSAY,cw +TSIT,cw +TWAC,cw +UARL,cw +UBIR,cw +UCHR,cw +UDEN,cw +UEUT,cw +UFRA,cw +UJER,cw +UNAR,cw +UNRS,cw +UNTH,cw +UNUR,cw +UOMI,cw +UPCE,cw +USHU,cw +USKE,cw +USTK,cw +UTRE,cw +VICT,cw +WILL,cw +WORC,cw +ZYMO,cw diff --git a/inst/extdata/configs/default_extrabreaks/README.md b/inst/extdata/configs/default_extrabreaks/README.md index 2ab9d55f..92870dd4 100644 --- a/inst/extdata/configs/default_extrabreaks/README.md +++ b/inst/extdata/configs/default_extrabreaks/README.md @@ -23,6 +23,7 @@ Not in this config: | `rules.yaml` | Built rules YAML (consumed by `frs_habitat_classify()`). Regenerate from `dimensions.csv` via `lnk_rules_build()` | | `dimensions.csv` | Source of `rules.yaml` — species × habitat biology encoded for NewGraph defaults. Source of truth is `inst/extdata/parameters_habitat_dimensions.csv` (copied in here on bundle assembly) | | `parameters_fresh.csv` | Per-species fresh overrides (spawn_gradient_min, observation_threshold, etc.) | +| `parameters_habitat_method.csv` | Per-watershed-group habitat size model, `cw` (channel width) or `mad` (mean annual discharge), handed to fresh by classify. All `cw`; a frozen copy of bcfishpass `parameters/example_newgraph`, not csv-synced, so moving a group to `mad` is a reviewed edit (update its `checksum` in `config.yaml`). Columns in `configs/dictionary_parameters_habitat_method.csv`. | | `overrides/` | Shared jurisdiction data — same barrier corrections, PSCIS status overrides, observation exclusions, habitat confirmations as the bcfishpass variant. These are BC-specific facts, not method choices. Redistributed under `LICENSE-bcfishpass` at the repo root. | The bundle is consumed via `lnk_config("default")` + `lnk_load_overrides(cfg)`. Project-experimental configs can declare `extends: default` to inherit this bundle and override specific entries (e.g. point a project's `user_barriers_definite` at a project-local CSV). diff --git a/inst/extdata/configs/default_extrabreaks/config.yaml b/inst/extdata/configs/default_extrabreaks/config.yaml index 67133996..0af00dbb 100644 --- a/inst/extdata/configs/default_extrabreaks/config.yaml +++ b/inst/extdata/configs/default_extrabreaks/config.yaml @@ -25,6 +25,8 @@ files: path: parameters_fresh.csv parameters_habitat_thresholds: path: parameters_habitat_thresholds.csv + parameters_habitat_method: + path: parameters_habitat_method.csv user_habitat_classification: source: bcfp path: overrides/user_habitat_classification.csv @@ -144,6 +146,13 @@ provenance: synced: '2026-09-25' checksum: sha256:7b904a18c29c724737ff24a27750bf05d7bfa281aa4fc1f7ac27c00238db042d shape_checksum: sha256:8de9bc1fcc2c703176a57d8a982bd26da1c49a885af3053f00078003a7e9283f + parameters_habitat_method.csv: + source: link (frozen copy; not csv-synced, link#286). Moving a group to mad is a reviewed edit here + derived_from: smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv + upstream_sha: 1fae4eae67284fbe7950dbdb043b589a20b428e2 + synced: '2026-10-01' + checksum: sha256:9802bb71bdd7f7d296e917d138567f096a69dab12dbc7c0f331e55206e5f6a0a + shape_checksum: sha256:cf285e3c204249a6b628424979ef780baa271add44cc8ea6512a44d6671a9eb5 overrides/user_habitat_classification.csv: source: https://github.com/smnorris/bcfishpass path: data/user_habitat_classification.csv diff --git a/inst/extdata/configs/default_extrabreaks/parameters_habitat_method.csv b/inst/extdata/configs/default_extrabreaks/parameters_habitat_method.csv new file mode 100644 index 00000000..be6081b3 --- /dev/null +++ b/inst/extdata/configs/default_extrabreaks/parameters_habitat_method.csv @@ -0,0 +1,189 @@ +watershed_group_code,model +ADMS,cw +ALBN,cw +ATNA,cw +BABL,cw +BABR,cw +BARR,cw +BBAR,cw +BELA,cw +BIGC,cw +BLAR,cw +BONP,cw +BOWR,cw +BRID,cw +BRKS,cw +BULK,cw +BULL,cw +CAMB,cw +CANO,cw +CARP,cw +CARR,cw +CHES,cw +CHIL,cw +CHIR,cw +CHWK,cw +CLAY,cw +CLRH,cw +CLWR,cw +COLR,cw +COMX,cw +COTR,cw +COWN,cw +CRKD,cw +DEAD,cw +DOGC,cw +DRIR,cw +DUNC,cw +ELKR,cw +EUCH,cw +EUCL,cw +FINA,cw +FINL,cw +FIRE,cw +FOXR,cw +FRAN,cw +FRCN,cw +GLAR,cw +GOLD,cw +GRAI,cw +GRNL,cw +GUIC,cw +HARR,cw +HERR,cw +HOLB,cw +HOMA,cw +HORS,cw +INGR,cw +INKR,cw +JENR,cw +JERV,cw +KEEC,cw +KETL,cw +KHOR,cw +KHTZ,cw +KINR,cw +KISP,cw +KITL,cw +KITR,cw +KLIN,cw +KLUM,cw +KNIG,cw +KOTL,cw +KOTR,cw +KSHR,cw +KTSU,cw +KUMR,cw +LARL,cw +LBIR,cw +LCHL,cw +LCHR,cw +LDEN,cw +LEUT,cw +LFRA,cw +LILL,cw +LISR,cw +LKEL,cw +LNAR,cw +LNIC,cw +LNRS,cw +LNTH,cw +LOMI,cw +LRDO,cw +LSAL,cw +LSKE,cw +LSTR,cw +LTRE,cw +MAHD,cw +MBNK,cw +MCGR,cw +MESC,cw +MESI,cw +MFRA,cw +MIDR,cw +MORI,cw +MORK,cw +MORR,cw +MSKE,cw +MSTR,cw +MURT,cw +MUSK,cw +NAHR,cw +NAKR,cw +NARC,cw +NASC,cw +NASR,cw +NATR,cw +NAZR,cw +NBNK,cw +NECL,cw +NECR,cw +NEVI,cw +NICL,cw +NIEL,cw +NIMP,cw +OKAN,cw +OSPK,cw +OWIK,cw +PARA,cw +PARK,cw +PARS,cw +PCEA,cw +PINE,cw +PORI,cw +QUES,cw +REVL,cw +SAJR,cw +SALM,cw +SALR,cw +SANJ,cw +SETN,cw +SEYM,cw +SHER,cw +SHUL,cw +SIML,cw +SKGT,cw +SLOC,cw +SMAR,cw +SQAM,cw +STHM,cw +STIR,cw +STUL,cw +STUR,cw +SUST,cw +SWIR,cw +TABR,cw +TAHR,cw +TAHS,cw +TAKL,cw +TASR,cw +TATR,cw +TAYR,cw +TESR,cw +THOM,cw +TOBA,cw +TOOD,cw +TSAY,cw +TSIT,cw +TWAC,cw +UARL,cw +UBIR,cw +UCHR,cw +UDEN,cw +UEUT,cw +UFRA,cw +UJER,cw +UNAR,cw +UNRS,cw +UNTH,cw +UNUR,cw +UOMI,cw +UPCE,cw +USHU,cw +USKE,cw +USTK,cw +UTRE,cw +VICT,cw +WILL,cw +WORC,cw +ZYMO,cw diff --git a/inst/extdata/configs/default_rearbreaks/README.md b/inst/extdata/configs/default_rearbreaks/README.md index 2ab9d55f..92870dd4 100644 --- a/inst/extdata/configs/default_rearbreaks/README.md +++ b/inst/extdata/configs/default_rearbreaks/README.md @@ -23,6 +23,7 @@ Not in this config: | `rules.yaml` | Built rules YAML (consumed by `frs_habitat_classify()`). Regenerate from `dimensions.csv` via `lnk_rules_build()` | | `dimensions.csv` | Source of `rules.yaml` — species × habitat biology encoded for NewGraph defaults. Source of truth is `inst/extdata/parameters_habitat_dimensions.csv` (copied in here on bundle assembly) | | `parameters_fresh.csv` | Per-species fresh overrides (spawn_gradient_min, observation_threshold, etc.) | +| `parameters_habitat_method.csv` | Per-watershed-group habitat size model, `cw` (channel width) or `mad` (mean annual discharge), handed to fresh by classify. All `cw`; a frozen copy of bcfishpass `parameters/example_newgraph`, not csv-synced, so moving a group to `mad` is a reviewed edit (update its `checksum` in `config.yaml`). Columns in `configs/dictionary_parameters_habitat_method.csv`. | | `overrides/` | Shared jurisdiction data — same barrier corrections, PSCIS status overrides, observation exclusions, habitat confirmations as the bcfishpass variant. These are BC-specific facts, not method choices. Redistributed under `LICENSE-bcfishpass` at the repo root. | The bundle is consumed via `lnk_config("default")` + `lnk_load_overrides(cfg)`. Project-experimental configs can declare `extends: default` to inherit this bundle and override specific entries (e.g. point a project's `user_barriers_definite` at a project-local CSV). diff --git a/inst/extdata/configs/default_rearbreaks/config.yaml b/inst/extdata/configs/default_rearbreaks/config.yaml index 956deb06..f6030206 100644 --- a/inst/extdata/configs/default_rearbreaks/config.yaml +++ b/inst/extdata/configs/default_rearbreaks/config.yaml @@ -21,6 +21,8 @@ files: path: parameters_fresh.csv parameters_habitat_thresholds: path: parameters_habitat_thresholds.csv + parameters_habitat_method: + path: parameters_habitat_method.csv user_habitat_classification: source: bcfp path: overrides/user_habitat_classification.csv @@ -139,6 +141,13 @@ provenance: synced: '2026-09-25' checksum: sha256:7b904a18c29c724737ff24a27750bf05d7bfa281aa4fc1f7ac27c00238db042d shape_checksum: sha256:8de9bc1fcc2c703176a57d8a982bd26da1c49a885af3053f00078003a7e9283f + parameters_habitat_method.csv: + source: link (frozen copy; not csv-synced, link#286). Moving a group to mad is a reviewed edit here + derived_from: smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv + upstream_sha: 1fae4eae67284fbe7950dbdb043b589a20b428e2 + synced: '2026-10-01' + checksum: sha256:9802bb71bdd7f7d296e917d138567f096a69dab12dbc7c0f331e55206e5f6a0a + shape_checksum: sha256:cf285e3c204249a6b628424979ef780baa271add44cc8ea6512a44d6671a9eb5 overrides/user_habitat_classification.csv: source: https://github.com/smnorris/bcfishpass path: data/user_habitat_classification.csv diff --git a/inst/extdata/configs/default_rearbreaks/parameters_habitat_method.csv b/inst/extdata/configs/default_rearbreaks/parameters_habitat_method.csv new file mode 100644 index 00000000..be6081b3 --- /dev/null +++ b/inst/extdata/configs/default_rearbreaks/parameters_habitat_method.csv @@ -0,0 +1,189 @@ +watershed_group_code,model +ADMS,cw +ALBN,cw +ATNA,cw +BABL,cw +BABR,cw +BARR,cw +BBAR,cw +BELA,cw +BIGC,cw +BLAR,cw +BONP,cw +BOWR,cw +BRID,cw +BRKS,cw +BULK,cw +BULL,cw +CAMB,cw +CANO,cw +CARP,cw +CARR,cw +CHES,cw +CHIL,cw +CHIR,cw +CHWK,cw +CLAY,cw +CLRH,cw +CLWR,cw +COLR,cw +COMX,cw +COTR,cw +COWN,cw +CRKD,cw +DEAD,cw +DOGC,cw +DRIR,cw +DUNC,cw +ELKR,cw +EUCH,cw +EUCL,cw +FINA,cw +FINL,cw +FIRE,cw +FOXR,cw +FRAN,cw +FRCN,cw +GLAR,cw +GOLD,cw +GRAI,cw +GRNL,cw +GUIC,cw +HARR,cw +HERR,cw +HOLB,cw +HOMA,cw +HORS,cw +INGR,cw +INKR,cw +JENR,cw +JERV,cw +KEEC,cw +KETL,cw +KHOR,cw +KHTZ,cw +KINR,cw +KISP,cw +KITL,cw +KITR,cw +KLIN,cw +KLUM,cw +KNIG,cw +KOTL,cw +KOTR,cw +KSHR,cw +KTSU,cw +KUMR,cw +LARL,cw +LBIR,cw +LCHL,cw +LCHR,cw +LDEN,cw +LEUT,cw +LFRA,cw +LILL,cw +LISR,cw +LKEL,cw +LNAR,cw +LNIC,cw +LNRS,cw +LNTH,cw +LOMI,cw +LRDO,cw +LSAL,cw +LSKE,cw +LSTR,cw +LTRE,cw +MAHD,cw +MBNK,cw +MCGR,cw +MESC,cw +MESI,cw +MFRA,cw +MIDR,cw +MORI,cw +MORK,cw +MORR,cw +MSKE,cw +MSTR,cw +MURT,cw +MUSK,cw +NAHR,cw +NAKR,cw +NARC,cw +NASC,cw +NASR,cw +NATR,cw +NAZR,cw +NBNK,cw +NECL,cw +NECR,cw +NEVI,cw +NICL,cw +NIEL,cw +NIMP,cw +OKAN,cw +OSPK,cw +OWIK,cw +PARA,cw +PARK,cw +PARS,cw +PCEA,cw +PINE,cw +PORI,cw +QUES,cw +REVL,cw +SAJR,cw +SALM,cw +SALR,cw +SANJ,cw +SETN,cw +SEYM,cw +SHER,cw +SHUL,cw +SIML,cw +SKGT,cw +SLOC,cw +SMAR,cw +SQAM,cw +STHM,cw +STIR,cw +STUL,cw +STUR,cw +SUST,cw +SWIR,cw +TABR,cw +TAHR,cw +TAHS,cw +TAKL,cw +TASR,cw +TATR,cw +TAYR,cw +TESR,cw +THOM,cw +TOBA,cw +TOOD,cw +TSAY,cw +TSIT,cw +TWAC,cw +UARL,cw +UBIR,cw +UCHR,cw +UDEN,cw +UEUT,cw +UFRA,cw +UJER,cw +UNAR,cw +UNRS,cw +UNTH,cw +UNUR,cw +UOMI,cw +UPCE,cw +USHU,cw +USKE,cw +USTK,cw +UTRE,cw +VICT,cw +WILL,cw +WORC,cw +ZYMO,cw diff --git a/inst/extdata/configs/default_tuned/README.md b/inst/extdata/configs/default_tuned/README.md index 281f0bf4..46738532 100644 --- a/inst/extdata/configs/default_tuned/README.md +++ b/inst/extdata/configs/default_tuned/README.md @@ -5,6 +5,7 @@ The `default` bundle with calibrated species habitat thresholds. It is a **thin* | File / key | Role | |------|------| | `config.yaml` | Manifest. `extends: default`; persist schema `fresh_default_tuned`; declares its own `parameters_habitat_thresholds` | +| `parameters_habitat_method.csv` (inherited) | Per-watershed-group `cw`/`mad` model, from `default` | | `parameters_habitat_thresholds.csv` | Per-species spawn/rear gradient max, channel-width min/max, MAD, lake-area floor and edge types, read by `lnk_pipeline_classify()` and `lnk_pipeline_connect()` | Everything else — `rules.yaml`, `dimensions.csv`, `parameters_fresh.csv`, `overrides/`, `break_order`, cluster settings — is inherited from `configs/default/` and resolves to that directory. `lnk_config("default_tuned")$chain` lists both. diff --git a/inst/extdata/configs/dictionary_parameters_habitat_method.csv b/inst/extdata/configs/dictionary_parameters_habitat_method.csv new file mode 100644 index 00000000..540a0f74 --- /dev/null +++ b/inst/extdata/configs/dictionary_parameters_habitat_method.csv @@ -0,0 +1,3 @@ +column,type,group,owner,consumed_by,default_when_absent,description,related +watershed_group_code,character,key,fresh,fresh/R/utils.R:887,required,"Watershed group code; row key, unique. A group the table does not list classifies on channel width (cw).",model +model,character,model,fresh,"fresh/R/utils.R:881; fresh/R/frs_habitat_classify.R:153",cw,"Habitat size model for the group: cw sizes streams by modelled channel width (channel_width against *_channel_width_min/max), mad by mean annual discharge (mad_m3s against *_mad_min/max in parameters_habitat_thresholds). Only cw and mad are accepted. Under mad, fresh follows bcfishpass: species with no MAD thresholds (BT, GR, KO, RB) get no stream habitat from inheriting rules, and the rule-level channel_width river-polygon bypass is ignored.",spawn_mad_min (parameters_habitat_thresholds) diff --git a/inst/extdata/configs/dictionary_parameters_habitat_thresholds.csv b/inst/extdata/configs/dictionary_parameters_habitat_thresholds.csv index 3ab3f8f6..52454df0 100644 --- a/inst/extdata/configs/dictionary_parameters_habitat_thresholds.csv +++ b/inst/extdata/configs/dictionary_parameters_habitat_thresholds.csv @@ -3,13 +3,13 @@ species_code,character,key,fresh,"fresh/R/frs_params.R:85; R/lnk_rules_build.R:1 spawn_gradient_max,numeric,gradient,fresh,"fresh/R/frs_habitat_classify.R:170; fresh/R/frs_habitat_predicates.R:89-95",required,"Upper stream-gradient bound for spawning, as a proportion (0.0449 = 4.49%). Inherited by every spawn rule in rules.yaml that sets no gradient of its own, paired with parameters_fresh spawn_gradient_min as the lower bound.",spawn_gradient_min (parameters_fresh) spawn_channel_width_min,numeric,channel_width,fresh,"fresh/R/frs_params.R:492-497; fresh/R/frs_habitat_predicates.R:89-95",NA (no width test),"Lower channel-width bound (m) for spawning. Inherited by spawn rules that set no channel_width; segments with NULL channel_width fail the BETWEEN test. River-polygon rules that set channel_width [0, 9999] (river_skip_cw_min in dimensions.csv) bypass it.",spawn_channel_width_max spawn_channel_width_max,numeric,channel_width,fresh,fresh/R/frs_params.R:493-496,Inf,"Upper channel-width bound (m) for spawning; 9999 means no limit. Only read when spawn_channel_width_min is set.",spawn_channel_width_min -spawn_mad_min,numeric,mad,fresh,fresh/R/frs_params.R:499-504,NA,"Lower mean-annual-discharge bound (m3/s) for spawning. Built into params ranges but not inherited by rules (fresh/R/utils.R:312 - MAD is rule-level only), and streams carry no mad_m3s (fresh#114), so it has no effect on link runs.",spawn_mad_max -spawn_mad_max,numeric,mad,fresh,fresh/R/frs_params.R:500-503,Inf,"Upper mean-annual-discharge bound (m3/s) for spawning. Same status as spawn_mad_min: carried, not applied on link's rules path.",spawn_mad_min +spawn_mad_min,numeric,mad,fresh,"fresh/R/frs_params.R:499-504; fresh/R/utils.R:334-340",NA,"Lower mean-annual-discharge bound (m3/s) for spawning. Applied only in watershed groups the bundle's parameters_habitat_method.csv puts on mad (#286), where it replaces spawn_channel_width_min for every spawn rule that sets no mad of its own; NA means the species gets no stream spawning there. No effect in cw groups, which is every group in every shipped bundle.",spawn_mad_max +spawn_mad_max,numeric,mad,fresh,"fresh/R/frs_params.R:500-503; fresh/R/utils.R:334-340",Inf,"Upper mean-annual-discharge bound (m3/s) for spawning. Same status as spawn_mad_min: applied only in mad groups.",spawn_mad_min rear_gradient_max,numeric,gradient,fresh,"fresh/R/frs_params.R:487-490; fresh/R/frs_habitat_predicates.R:115-118",required,"Upper stream-gradient bound for rearing, as a proportion. Inherited by rear rules that set no gradient (lower bound fixed at 0). Lake and wetland rules never test gradient (fresh/R/utils.R:225-233).",- rear_channel_width_min,numeric,channel_width,fresh,"fresh/R/frs_params.R:492-497; fresh/R/frs_habitat_predicates.R:115-118",NA (no width test),"Lower channel-width bound (m) for rearing. Inherited by rear rules that set no channel_width; NULL channel_width fails. River-polygon rules with channel_width [0, 9999] bypass it.",rear_channel_width_max rear_channel_width_max,numeric,channel_width,fresh,fresh/R/frs_params.R:493-496,Inf,"Upper channel-width bound (m) for rearing; 9999 means no limit. Only read when rear_channel_width_min is set.",rear_channel_width_min -rear_mad_min,numeric,mad,fresh,fresh/R/frs_params.R:499-504,NA,"Lower mean-annual-discharge bound (m3/s) for rearing. Carried, not applied on link's rules path (see spawn_mad_min).",rear_mad_max -rear_mad_max,numeric,mad,fresh,fresh/R/frs_params.R:500-503,Inf,"Upper mean-annual-discharge bound (m3/s) for rearing. Carried, not applied on link's rules path.",rear_mad_min +rear_mad_min,numeric,mad,fresh,"fresh/R/frs_params.R:499-504; fresh/R/utils.R:334-340",NA,"Lower mean-annual-discharge bound (m3/s) for rearing. Applied only in mad groups (see spawn_mad_min); NA means no stream rearing there.",rear_mad_max +rear_mad_max,numeric,mad,fresh,"fresh/R/frs_params.R:500-503; fresh/R/utils.R:334-340",Inf,"Upper mean-annual-discharge bound (m3/s) for rearing. Applied only in mad groups.",rear_mad_min rear_lake_ha_min,numeric,waterbody,fresh,"R/lnk_rules_build.R:281; R/lnk_rules_build.R:376",NA (no lake-area floor),"Minimum lake area (ha) for lake rearing. Baked into rules.yaml by lnk_rules_build() as the lake rule's lake_ha_min, so a change here only takes effect after the bundle's rules.yaml is rebuilt (data-raw/build_rules.R).",rear_lake (dimensions.csv) spawn_edge_types,character,edge,fresh,fresh/R/frs_habitat_predicates.R:106,NA (all edge types),"Comma-separated edge-type categories for spawning (fresh#104). Only read on fresh's no-rules fallback path; link always supplies rules.yaml, whose edge_types_explicit take precedence.",rear_edge_types rear_edge_types,character,edge,fresh,fresh/R/frs_habitat_predicates.R:143,NA (all edge types),"Comma-separated edge-type categories for rearing (fresh#104). Only read on fresh's no-rules fallback path; not used by link runs.",spawn_edge_types diff --git a/man/lnk_preflight_vintage.Rd b/man/lnk_preflight_vintage.Rd index 52523110..e9204198 100644 --- a/man/lnk_preflight_vintage.Rd +++ b/man/lnk_preflight_vintage.Rd @@ -46,7 +46,7 @@ May inputs and three on August inputs, produced one consolidated table set, and said nothing about it anywhere (link#246). } \details{ -The other seven tables in \code{.lnk_input_primitives()} are bulk-restored +The FWA tables in \code{.lnk_input_primitives()} are bulk-restored FWA. They are never \code{ANALYZE}d, so they carry no vintage at all and are not an axis this can measure — including them would mean every host failing forever on data that is not the staleness risk. diff --git a/planning/active/findings.md b/planning/active/findings.md index 69f47389..d475d6cb 100644 --- a/planning/active/findings.md +++ b/planning/active/findings.md @@ -44,6 +44,83 @@ Thresholds in both currencies, with verbatim quotes, are in NewGraphEnvironment/ still reads `PG_*_SHARE` first. `lnk_db_conn()`'s roxygen claimed to work "identically to `frs_db_conn()`", which is now false, so that is corrected too. +## Phase 2 — method table (2026-10-01) + +- `smnorris/bcfishpass@f8db4b9` `parameters/example_newgraph/parameters_habitat_method.csv` + (last touched `1fae4ea`) has **188 groups, all `cw`**, unquoted. fresh's + `inst/extdata/parameters_habitat_method.csv` has the same rows, quoted, **minus PINE + (187)**. That makes no difference to the output, since fresh treats a group its table + does not list as `cw`. All four base bundles carry the bcfp copy, which is + byte-identical across them: `sha256:9802bb71…`. +- The checksum and shape_checksum come from `.lnk_shape_fingerprint()`, and + `lnk_config_verify()` reports no drift on all five bundles. `audit_configs.R` reports + "No findings". +- **`config_hash` changes for every shipped bundle** (new declared file). For a custom + bundle with no method file it changes too, because fresh's copy is now hashed. Mutation + check: dropping the fallback digest turns `test-lnk_log.R:1234` red. + +## Phase 3 — classify (2026-10-01) + +- `.frs_run_connectivity` is not method-aware: it clusters on gradient and adjacency. + `lnk_pipeline_connect` is unchanged. +- `method_csv` is read with `colClasses = "character", na.strings = character(0)`, so a + group coded `NA` stays a code, which read.csv's default would not do. + +## Plan review + code-check round 2 triage (2026-10-01) + +Plan review: `planning/active/review-plan.md`. Round 2: `review-p23-round2.md`. **Both +independently found B1**, which was the one real bug. + +| finding | disposition | +|---|---| +| B1 csv-sync would auto-merge the method CSV | **fixed.** Source is now `link (frozen copy; not csv-synced…)`, with `derived_from: smnorris/bcfishpass@1fae4ea …` and `upstream_sha` = the last commit touching the file. Test: `test-lnk_config.R` "the method table is frozen". | +| G1 stale mad docs | **fixed.** Four `*_mad_*` rows in the thresholds dictionary (with fresh `utils.R:334-340` refs), the RUNBOOK §7 bullet, CLAUDE.md, and all five bundle READMEs. | +| G2 discharge not fingerprinted | **fixed.** Added to `.lnk_input_primitives()` (`rep(fwapg, 8L)`). The "seven FWA tables" wording in the vintage doc and test now says "the FWA tables". The DB-gated existence test ran (SKIP 0) and finds it. | +| G3 stream-order bypass under mad | **fixed.** Skipped when the AOI's model is `mad`. Tests cover the cw/mad pair. | +| G4 connect reason | recorded precisely in RUNBOOK. No code change. | +| G5 join / persist-shape tests | **added** (`test-lnk_pipeline_prepare.R`). | +| G6 cross-package contract | **added** (`test-dictionaries.R`). | +| G7 audit section | **scoped out.** `test-dictionaries.R` already checks domain, duplicates and missing codes per bundle, plus the fresh header contract, and it runs in CI where `data-raw/audit_configs.R` does not. | +| G8 follow-up draft | updated (persist has no `mad_m3s`). The stale v0.33.0 rationale was fixed in Phase 1. | +| O1 isolate column + fresh bump | **done.** A0/A/A_nocol/B/B2 (Phase 4). | +| O2 thin bundle → `fresh` schema | avoided. The mad runs pass `method_csv` on scratch schemas; nothing persists. | +| O3 Phase 1 boxes | already flipped in `bd2f8ff`; the review read a pre-commit tree. | +| O4 cypher re-prep | in RUNBOOK and the PR body / NEWS draft. | +| A3 `loaded` vs path | documented in the `method_csv` roxygen. | +| AC1–AC3 invariants | `mad_check.R`: classify only, overlay off, stream vs waterbody split. | +| AC4 verify loop | **added** (`test-lnk_config.R`). | + +## Follow-up issue draft (NOT filed — needs body review) + +**Title:** `lnk_habitat_validate()` scores `mad` watershed groups as if they were `cw` + +> **If done:** observation validation reports the right miss reasons for groups a bundle +> puts on discharge. **If never:** a `mad` group's "missed for width" counts are computed +> against channel width the classification never used. Nothing is affected until a +> bundle actually sets a group to `mad`, and none does today. +> +> Since #286, `lnk_pipeline_classify()` passes the bundle's +> `parameters_habitat_method.csv` to fresh, and a `mad` group classifies on `mad_m3s`. +> `lnk_habitat_validate()` still builds its predicates with +> `fresh::frs_habitat_predicates(spp)`, which is the cw model +> (`R/lnk_habitat_validate.R:823`). It then relaxes width by rewriting `s.channel_width` +> (`.lnk_hv_relax()`, `:772-786`), with minimums taken from +> `ranges$$channel_width` (`.lnk_hv_stage_min()`, `:763`). On a `mad` group: +> - the predicate is not the one that classified the segment, so `pred_*` can disagree +> with the persisted `spawning`/`rearing`; +> - width relaxation rewrites a column the mad predicate does not reference; +> - `width_null` (`.lnk_hv_miss_reason()`) tests `channel_width`, where it should test +> `mad_m3s`. The persist does not carry `mad_m3s` (decided in #286), so the validator +> would join it from `whse_basemapping.fwa_stream_networks_discharge` on +> `linear_feature_id`. +> - The stream-order rearing bypass is skipped for `mad` groups (#286), so a miss +> reason must not credit it there. +> +> fresh >= 0.35.0 has `frs_habitat_predicates(model = "mad")`. Resolve each scored +> group's model from the bundle's method table, build the predicates per model, and relax +> `s.mad_m3s` for mad groups. `test-lnk_habitat_validate.R` "the predicate call stays on +> the channel-width model" pins today's behaviour and should change with it. + ## Errors Encountered | Error | Resolution | diff --git a/planning/active/progress.md b/planning/active/progress.md index d0d332df..8ea9d225 100644 --- a/planning/active/progress.md +++ b/planning/active/progress.md @@ -7,3 +7,4 @@ - Scaffolded PWF baseline from issue #286 with approved phases - Next: start Phase 1 - Phase 1: fresh pin -> v0.36.2 (floor 0.35.0); `lnk_preflight_fresh(required_formals=)` asserts `frs_habitat_classify(params_method)`; vignette script off `frs_db_conn()` (fresh 0.36.0 precedence flip). Code-check round 1 clean; it flagged stale text (validator/test comments citing v0.33.0, CLAUDE.md `frs_db_conn` line, a skip message), fixed in the same commit +- Phase 2: method CSV in the four base bundles (frozen, not csv-synced: B1 from the plan review and code-check round 2, found by both independently), resolver + config_hash fallback, dictionary + tests, discharge in `log_input` primitives, stale `*_mad_*` docs fixed, verify-clean loop over every bundle. Code-check rounds 2 (B1) and 3 (clean) diff --git a/planning/active/review-p23-round2.md b/planning/active/review-p23-round2.md new file mode 100644 index 00000000..7dfa0e72 --- /dev/null +++ b/planning/active/review-p23-round2.md @@ -0,0 +1,102 @@ +# Review — #286 phases 2-3, round 2 + +Reviewed: staged diff (`diff_p23.patch`) against the working tree, fresh v0.36.2 +(installed, RemoteSha e3a37f0) and fresh source at c341a59, db_newgraph @ 88bb47c. + +## Findings + +- **[severity: bug]** `inst/extdata/configs/bcfishpass/config.yaml:137-143` (and the + identical entry in `default/config.yaml:147-153`) — the new provenance entry declares + `source: https://github.com/smnorris/bcfishpass`, which enrolls the "frozen" method + table in the weekly csv-sync, and that sync auto-merges. + - `data-raw/sync_bcfishpass_csvs.R:120-122,149-150` picks every bcfishpass-bundle + provenance entry whose `source` is exactly that URL, then fetches + `csvs/` = `csvs/parameters_habitat_method.csv` from s3://fresh-bc. + - That key exists and is refreshed weekly: `db_newgraph/jobs/dump_bcfishpass_csvs` + (`PARAM_CSVS=( parameters_habitat_method.csv … )`, from + `parameters/example_newgraph/` at the latest ng-prod SHA). I fetched it with curl: + HTTP 200, sha256 `9802bb71…`. That is byte-identical to the shipped copy today, so + nothing happens yet. + - The first time upstream edits example_newgraph's method table (other + `parameters/example_*` dirs already carry `mad` rows, e.g. HORS and LNIC), the sync: + - writes the new bytes into **both** `bcfishpass/` and `default/` (`:251-264`, which + writes `BUNDLE_BCFP` and `BUNDLE_DEF` unconditionally) and rewrites both + provenance blocks. The drift is byte drift (the shape is unchanged), so + `sync-bcfishpass-csvs.yml:143` runs `gh pr merge` without review or R CMD check. + - Effects: + - `default`, which `default_tuned` inherits, silently moves groups to `mad`. Under + fresh's mad semantics, BT/GR/KO/RB get no stream habitat in those groups. + - `default_extrabreaks` and `default_rearbreaks` are not touched, so they quietly + diverge from `default`. + - `config_hash` moves for the bundles that changed. + - `test-lnk_pipeline_classify.R` ("passes a shipped bundle's own method table", + `expect_true(all(pm$model == "cw"))`) goes red on main after the fact. + - This contradicts the stated decision that each base bundle carries a **frozen** + copy. The thresholds entry avoided this by declaring + `source: https://github.com/NewGraphEnvironment/fresh` with the bcfishpass origin in + `derived_from:`. + - Fix: give the method entries a non-matching `source:` (e.g. `link (frozen copy; + link#286)`) plus `derived_from: smnorris/bcfishpass@f8db4b9 + parameters/example_newgraph/parameters_habitat_method.csv`, in all four bundles. + Changing `bcfishpass/config.yaml` alone stops the sync, because the script reads only + that bundle's provenance. The alternative is to decide that the bcfishpass bundle + tracks upstream for parity, but then the sync script must stop writing `default`. + +## Checked and clean + +1. **mad_m3s leak from the working streams table.** + - `lnk_pipeline_persist.R:55` inserts with the explicit `cols_streams` list, so the + new column is never persisted. + - `frs_break_apply` carries every writable column to split children + (`fresh/R/frs_break.R:660-700`), so `mad_m3s` survives segmentation, which is what + the mad path needs. + - `lnk_access.R:175` `CREATE TABLE … SELECT *` reads the *persist* streams. It is + harmless either way. + - The `habitat_variants_build.R` digests hash `streams_habitat` rows and + `streams(id_segment, length_metre)`, not working-streams columns, so they are + unaffected. + - Old kept working schemas without `mad_m3s` re-classify fine while every group is + `cw`. fresh checks for the column only when a group is `mad`. +2. **Missing discharge table.** + - Local docker fwapg has `whse_basemapping.fwa_stream_networks_discharge`: a + table, 2,716,652 rows, a unique pkey on `linear_feature_id`, and `mad_m3s` is + double precision. So the `UPDATE … FROM` cannot fan out. + - fresh's docker `load.sh` has loaded discharge since its first commit (README stage + 2), so cyphers built from it carry it as well. + - If a DB lacked it, `frs_col_join` would add a `text` column (the + information_schema lookup returns nothing) and the `UPDATE … FROM` would then + **error**. That is a loud failure, not a silent one. + - The bcfishpass tunnel is not a modelling target: it lacks the `fresh.*` + primitives anyway. +3. **Other classify entry points.** + - The only `frs_habitat_classify` call is in `lnk_pipeline_classify`. + - `data-raw/compare_adms.R` calls `fresh::frs_habitat()` directly. It is a legacy + experiment, and fresh's own default (all `cw`) applies. + - `habitat_variants_build.R`, `exp_gradient_extra_breaks.R` and the two #282 logs + scripts all go through `lnk_pipeline_classify`. The variant bundles use + `extends: default`, so they inherit the method table. +4. **config_hash change.** + - Nothing gates on a stored hash. + - `.lnk_log_config_snapshot` keys on the hash and simply inserts a new snapshot. + - `lnk_preflight_parity` compares hosts within one run, all on the same code. + - The `habitat_variants_build.R` resume gate keys on `link_sha` and `link_dirty` + (and so rebuilds at a new HEAD regardless), not on the hash. + - `lnk_config_verify` reports no drift and no missing file for all five bundles, + `default_tuned` included. +5. **read.csv.** + - A UTF-8 BOM is stripped on this host and the header comes out clean. + - CRLF parses. + - A blank `model` cell comes through as `""`, which fresh rejects loudly. + - A quoted header (fresh's own copy is fully quoted) parses. + - A header-only file gives 0 rows, so every group is `cw`, as designed. + - `"NA"` stays a string. + +## Notes (not findings) + +- `.lnk_input_primitives()` (`R/lnk_log.R:200`) does not fingerprint + `whse_basemapping.fwa_stream_networks_discharge`. That is irrelevant while every + bundle is all-`cw`. Once a bundle puts a group on `mad`, the discharge vintage feeds + output without a `log_input` row. +- fresh's shipped copy (sha256 `6b5414ac…`, quoted, 187 rows) is not byte-identical to + the bundles' copies (188 rows). That is fine: it only matters for the fallback path, + which is hashed by content. diff --git a/planning/active/review-plan.md b/planning/active/review-plan.md new file mode 100644 index 00000000..d51adad6 --- /dev/null +++ b/planning/active/review-plan.md @@ -0,0 +1,25 @@ +# Plan review — #286 (Plan agent, 2026-10-01) + +Read-only agent; findings arrived as reply text and are recorded here by the parent. +Triage and dispositions are in findings.md, "Plan review triage". + +- **B1 (Blocker):** the provenance `source: https://github.com/smnorris/bcfishpass` puts the method CSV under weekly csv-sync (`data-raw/sync_bcfishpass_csvs.R:121-122`, an exact match on source). Byte drift auto-merges, so a ng-prod `cw`→`mad` flip would land unreviewed, and only in `bcfishpass` + `default` (`:251-260`). Freeze it the way the thresholds are frozen (non-bcfp `source` + `derived_from`). upstream_sha should be `1fae4ea` (last touch) vs the HEAD `f8db4b9` that was recorded. +- **G1:** stale docs once mad is live: `dictionary_parameters_habitat_thresholds.csv` `*_mad_*` rows ("no effect on link runs"); RUNBOOK §7; CLAUDE.md:78-79; bundle README file tables. +- **G2:** `.lnk_input_primitives()` (`R/lnk_log.R:186-216`) does not fingerprint `fwa_stream_networks_discharge`. +- **G3:** the `frs_order_child` stream-order bypass in classify runs regardless of model; bcfp applies it only in the cw branch (`load_habitat_linear_co.sql:97-110`). Latent: all shipped bundles have `rear_stream_order_bypass = no`. +- **G4:** connect: `.frs_connected_waterbody` phase 3 reads `s.channel_width >= spawn_connected$channel_width_min` (fresh `R/frs_habitat.R:1643-1646`). Inert (0 for SK/KO everywhere); conclusion holds but the reason needs to be precise. +- **G5:** no test pins the discharge join args, or that `cols_streams` excludes `mad_m3s`. +- **G6:** no cross-package test of dictionary columns vs fresh's method CSV header. +- **G7:** `audit_configs.R` has no method-table section. +- **G8:** follow-up draft should note that persist has no `mad_m3s`, and the stale v0.33.0 rationale (already fixed in Phase 1). +- **O1:** the no-change proof isolates neither the mad_m3s column nor the fresh 0.33.0→0.36.2 bump. Run HEAD classify under fresh 0.33.0 (separate libpath) and 0.36.2, then branch: A0 = A = B. +- **O2:** a thin bundle extending `default` inherits `pipeline.schema: fresh`; do the mad check via `method_csv` on a scratch schema. No-change before mad on the same schema. +- **O3:** Phase 1 checkboxes. Already flipped in commit bd2f8ff (the review read a pre-commit tree). +- **O4:** cyphers need re-prep onto fresh >= 0.35 before the next dispatch; the preflight hard-fails otherwise. Put this in NEWS + RUNBOOK. +- **A1–A8:** checked. 188 vs 187 (PINE plus quoting); discharge 150 groups by row presence vs 123 in fresh NEWS (pick a WSG by non-NULL share); `loaded$parameters_habitat_method` is read but unused, while classify re-reads the path (document); the join fails on a DB without the discharge table (cw groups too); mad_m3s survives breaks and does not leak to persist; verify requires `checksum`; every base bundle's config_hash changes; no existing tests break. +- **S1–S3:** no other call site needs params_method (`data-raw/compare_adms.R` frs_habitat is legacy all-cw); the validate follow-up is correctly deferred; no `.lnk_fresh_required()` change. +- **AC1:** the overlay contaminates "BT gets no stream habitat"; measure before overlay/connect, overlay off, stream segments only. +- **AC2:** replace "differs from cw" with invariants: every stream spawn/rear segment has mad_m3s within [min, max]; no NULL-mad segment is stream habitat. +- **AC3:** BULK mad: zero stream spawn/rear for every species, overlay off; lake/wetland may be non-zero. +- **AC4:** loop test asserting `lnk_config_verify` is clean for every shipped bundle. +- **AC5:** a per-species digest after connect, with the same query on both sides. diff --git a/planning/active/review-round3.md b/planning/active/review-round3.md new file mode 100644 index 00000000..e00d0d07 --- /dev/null +++ b/planning/active/review-round3.md @@ -0,0 +1,101 @@ +# Code-check round 3: #286 staged diff + +## Clean + +No issues found. + +## Mechanism behind the round-2 bug + +**Metadata doubling as a selector.** The provenance `source:` field reads as a +description, but `data-raw/sync_bcfishpass_csvs.R` (`is_bcfp_sourced()`, line 121) +uses it as an enrolment key: any entry whose `source` equals the bcfishpass URL joins +the weekly auto-merging sync. So the set of files the sync manages is never declared +anywhere. It is whatever happens to match a string. More generally: several +hand-kept lists (sync targets, input primitives, bundle files, dictionaries, persist +columns) each have to agree with the bundle, and nothing checks that they do. The +cw-only half of the question has the same shape: link code that assumes the size +model without asking the method table. + +## Every place that mechanism reaches, and what I found + +**Provenance used as a selector** +- `sync_bcfishpass_csvs.R`: the only reader that keys on `source`; it never reads + `upstream_sha` or `derived_from`. The new entry's `source` does not match, and + `test-lnk_config.R` pins that. +- `lnk_config_verify.R` reads `checksum` / `shape_checksum` only. All 5 bundles + verify clean, `default_tuned` included through inheritance. +- `.lnk_config_hash` builds paths from `cfg$files` plus provenance keys. The new + file is hashed by relative name, and for `default_tuned` as + `extends:default/...`. +- `regen_provenance.R` uses a fixed one-off list and does not touch this file. + +**Lists of bundle files** +- `lnk_load_overrides` loads every `cfg$files` entry: 16 for bcfishpass, 18 for + default, the new file included (smoke-tested by `audit_configs.R`). +- Nothing reads `loaded$parameters_habitat_method`, so read.csv's default `"NA"` → NA + there is inert. Classify reads the path with `na.strings = character(0)`. +- `audit_configs.R` ran to completion with "No findings", exit 0, and left the tree + unchanged. +- `default_tuned` resolves its method table to `configs/default/` (checked). +- `habitat_variants_build.R` variant bundles `extends: default`, so they inherit + the table. + +**Lists of input primitives** +- `.lnk_input_primitives` now holds 12 tables. +- `.lnk_vintage_primitives` filters on `source` not starting with `fwapg/`, so it + still returns 4 tables. +- `.lnk_log_inputs` iterates the list; no row count is hardcoded in R/, tests, + `study_area_verify.sql` or shell scripts. +- fwapg `load.sh:154` loads `fwa_stream_networks_discharge` alongside channel width, + so cypher DBs built the standard way carry it. + +**Persist shape** +- `lnk_pipeline_persist` inserts named `cols_streams` columns, never `SELECT *`. +- fresh's break step carries columns dynamically (`frs_break.R:668-677`), so + `mad_m3s` survives breaking on the working table. +- Working schemas built before this branch and reused by + `habitat_variants_build.R` lack `mad_m3s`. That is harmless while every group is + `cw`: fresh only checks for the column when a group resolves to `mad`. + +**The bundle copy against upstream** +- The bundle CSV is byte-identical to `smnorris/bcfishpass@1fae4ea` + `parameters/example_newgraph/parameters_habitat_method.csv` (sha256 `9802bb71…`, + matching the declared checksum). + +**cw-only behaviour applied regardless of the group's model** +- `frs_order_child` bypass: now skipped for a `mad` aoi. The skip decision is + consistent with fresh: + - Both use exact, case-sensitive matching. + - Both treat an unlisted group as `cw`. + - The working streams table is filtered to `watershed_group_code = aoi`, so + fresh's per-row model and link's per-aoi model cannot disagree. +- Connectivity: `.frs_run_connectivity` and the cluster helpers contain no + width or model reference. `.frs_connected_waterbody` adds a width test only when + `channel_width_min > 0`, and every rules.yaml has `0.0`, so it is inert as the + RUNBOOK says. +- `lnk_rules_build` river-polygon `channel_width [0, 9999]`: rule-level, and + fresh ignores it under `mad` (documented). +- Break sources in every bundle are gradient, observation, crossing and habitat + endpoints. None is width-based. +- `lnk_habitat_validate`: cw-only, accepted, not re-flagged. +- `lnk_pipeline_pscis_build` `downstream_channel_width` is PSCIS field data used + to match crossings, not the size model. + +**Malformed-input paths** +These each fail loud, through fresh's `.frs_habitat_models()` validation or the +`method_csv not found` stop, before the bypass decision runs: +- missing `model` column +- blank model +- whitespace in a value +- duplicate group rows +- BOM header +- `method_csv = NA` +- old fresh with no copy of the table + +## Informational, not a link defect + +fresh's own shipped `parameters_habitat_method.csv`, which custom bundles fall back +to, lists 187 groups. It lacks `PINE`, which upstream example_newgraph now carries. +Because an unlisted group classifies as `cw`, the fallback behaves identically. The +copy is quoted, so it is not byte-comparable to the bundles. It is worth a fresh-side +refresh some day, and nothing in link depends on it. diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md index 853060f6..4e76a397 100644 --- a/planning/active/task_plan.md +++ b/planning/active/task_plan.md @@ -36,12 +36,12 @@ needs adding there. - [x] Check `data-raw/wsg_vignette_data.R`'s bare `fresh::frs_db_conn()` against the 0.36.0 behaviour change, and note the result in findings ## Phase 2: method table as bundle data -- [ ] Add `parameters_habitat_method.csv` to `bcfishpass`, `default`, `default_extrabreaks` and `default_rearbreaks`: a frozen copy of `smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv` (188 groups, all `cw`). Diff it against fresh's bundled copy first and record the result. `default_tuned` inherits it through `extends` -- [ ] For each, add a `files: parameters_habitat_method:` entry and a provenance block (source, upstream_sha, synced, checksum, shape_checksum), the same as the thresholds entries -- [ ] Add a `.lnk_habitat_method_csv(cfg)` resolver in `R/lnk_config.R`, next to `.lnk_habitat_thresholds_csv()`: the bundle's path, else fresh's copy with a message -- [ ] `config_hash` (`R/lnk_log.R` ~L73/117): hash fresh's fallback as `fresh:parameters_habitat_method.csv` when the bundle declares none -- [ ] Add `inst/extdata/configs/dictionary_parameters_habitat_method.csv` and its shape/coverage/no-absent-column tests in `test-dictionaries.R`, following the #282 block -- [ ] `lnk_config_verify` / audit pass clean on all bundles +- [x] Add `parameters_habitat_method.csv` to `bcfishpass`, `default`, `default_extrabreaks` and `default_rearbreaks`: a frozen copy of `smnorris/bcfishpass@1fae4ea parameters/example_newgraph/parameters_habitat_method.csv` (188 groups, all `cw`). Diff it against fresh's bundled copy first and record the result. `default_tuned` inherits it through `extends` +- [x] For each, add a `files: parameters_habitat_method:` entry and a provenance block (source, upstream_sha, synced, checksum, shape_checksum), the same as the thresholds entries +- [x] Add a `.lnk_habitat_method_csv(cfg)` resolver in `R/lnk_config.R`, next to `.lnk_habitat_thresholds_csv()`: the bundle's path, else fresh's copy with a message +- [x] `config_hash` (`R/lnk_log.R` ~L73/117): hash fresh's fallback as `fresh:parameters_habitat_method.csv` when the bundle declares none +- [x] Add `inst/extdata/configs/dictionary_parameters_habitat_method.csv` and its shape/coverage/no-absent-column tests in `test-dictionaries.R`, following the #282 block +- [x] `lnk_config_verify` / audit pass clean on all bundles ## Phase 3: thread through classify + `mad_m3s` on working streams - [ ] `.lnk_pipeline_prep_network()`: add `fresh::frs_col_join(..., from = "whse_basemapping.fwa_stream_networks_discharge", cols = "mad_m3s", by = "linear_feature_id")` next to the channel_width join. Check that `frs_break_apply` carries it through splits diff --git a/tests/testthat/test-dictionaries.R b/tests/testthat/test-dictionaries.R index 7e2e8b34..edcfcba6 100644 --- a/tests/testthat/test-dictionaries.R +++ b/tests/testthat/test-dictionaries.R @@ -184,6 +184,59 @@ test_that("every bundle declares its own parameters_habitat_thresholds", { } }) +# -- dictionary_parameters_habitat_method (#286) ------------------------------ + +test_that("dictionary_parameters_habitat_method has the expected shape", { + d <- dict_read("parameters_habitat_method") + + expect_true(all(c("column", "type", "group", "owner", "consumed_by", + "default_when_absent", "description", + "related") %in% names(d))) + expect_false(any(duplicated(d$column))) + expect_true(all(filled(d$column))) + expect_true(all(filled(d$description))) + expect_true(all(filled(d$consumed_by))) + expect_true(all(d$owner %in% c("fresh", "link"))) +}) + +test_that("dictionary_parameters_habitat_method matches every bundle's CSV", { + d <- dict_read("parameters_habitat_method") + for (b in bundle_names()) { + expect_setequal(bundle_cols(b, "parameters_habitat_method"), d$column) + } +}) + +test_that("every bundle declares its own parameters_habitat_method", { + # As for thresholds: the fallback to fresh's copy is for custom bundles. + for (b in bundle_names()) { + expect_false( + is.null(lnk_config(b)$files$parameters_habitat_method), + info = sprintf("bundle %s declares no parameters_habitat_method", b)) + } +}) + +test_that("every bundle's method table is one valid row per group", { + # fresh rejects these at classify time, after a WSG's prepare and break + # phases have already run; catch them at the bundle. + for (b in bundle_names()) { + m <- read_csv_plain(lnk_config(b)$files$parameters_habitat_method$path) + expect_true(all(m$model %in% c("cw", "mad")), info = b) + expect_false(any(duplicated(m$watershed_group_code)), info = b) + expect_true(all(filled(m$watershed_group_code)), info = b) + } +}) + +test_that("dictionary_parameters_habitat_method matches fresh's method table", { + # Cross-package: the table is handed to fresh as params_method, so its + # columns are fresh's contract. + skip_if_not_installed("fresh") + canonical <- system.file("extdata", "parameters_habitat_method.csv", + package = "fresh") + skip_if(!nzchar(canonical), "fresh's parameters_habitat_method.csv not found") + expect_setequal(dict_read("parameters_habitat_method")$column, + names(read_csv_plain(canonical))) +}) + # -- dictionary_species_pooling / dictionary_species_groups (#290) ----------- # Only bundles that declare the file carry it; a bundle without a tracker pools diff --git a/tests/testthat/test-lnk_config.R b/tests/testthat/test-lnk_config.R index 65e3a8ed..29a3154a 100644 --- a/tests/testthat/test-lnk_config.R +++ b/tests/testthat/test-lnk_config.R @@ -337,6 +337,31 @@ test_that("inherited provenance verifies against the parent's files", { expect_false(any(v$byte_drift | v$shape_drift)) }) +test_that("every shipped bundle's provenance verifies clean", { + bundles <- basename(list.dirs(system.file("extdata", "configs", + package = "link"), + recursive = FALSE)) + expect_gt(length(bundles), 0L) + for (b in bundles) { + v <- lnk_config_verify(lnk_config(b)) + expect_false(any(v$missing), info = b) + expect_false(any(v$byte_drift | v$shape_drift), info = b) + } +}) + +test_that("the method table is frozen, not enrolled in csv-sync (#286)", { + # data-raw/sync_bcfishpass_csvs.R syncs (and auto-merges) every bcfishpass + # bundle entry whose source is exactly the bcfishpass repo URL. A cw -> mad + # flip upstream would then change the model of `default` with no review. + for (b in c("bcfishpass", "default", "default_extrabreaks", + "default_rearbreaks")) { + src <- lnk_config(b)$provenance[["parameters_habitat_method.csv"]]$source + expect_false(is.null(src), info = b) + expect_false(identical(src, "https://github.com/smnorris/bcfishpass"), + info = b) + } +}) + test_that("a tuned threshold reaches frs_params; bcfishpass stays put", { skip_if_not_installed("fresh") bcfp_before <- lnk_config_verify(lnk_config("bcfishpass")) diff --git a/tests/testthat/test-lnk_log.R b/tests/testthat/test-lnk_log.R index 325d4851..7e4b239e 100644 --- a/tests/testthat/test-lnk_log.R +++ b/tests/testthat/test-lnk_log.R @@ -1221,6 +1221,38 @@ test_that(".lnk_config_hash ignores the fallback when the bundle declares its ow expect_match(.lnk_config_hash(cfg), "^sha256:") }) +test_that(".lnk_config_hash covers fresh's method table when the bundle has none", { + # #286: a fresh upgrade that moves a group to mad must move the hash. + cfg <- lnk_config("default") + cfg$files$parameters_habitat_method <- NULL + fake <- withr::local_tempfile(fileext = ".csv") + writeLines("watershed_group_code,model\nADMS,cw", fake) + local_mocked_bindings(.lnk_habitat_method_csv = function(cfg) fake) + h1 <- .lnk_config_hash(cfg) + writeLines("watershed_group_code,model\nADMS,mad", fake) + h2 <- .lnk_config_hash(cfg) + expect_false(identical(h1, h2)) +}) + +test_that(".lnk_config_hash ignores the method fallback when the bundle declares one", { + cfg <- lnk_config("default") + local_mocked_bindings(.lnk_habitat_method_csv = function(cfg) { + stop("fallback consulted for a bundle that declares its own") + }) + expect_match(.lnk_config_hash(cfg), "^sha256:") +}) + +test_that(".lnk_config_hash moves when a bundle's own method table moves", { + cfg <- lnk_config("default") + own <- withr::local_tempfile(fileext = ".csv") + writeLines("watershed_group_code,model\nADMS,cw", own) + cfg$files$parameters_habitat_method <- list(path = own) + h1 <- .lnk_config_hash(cfg) + writeLines("watershed_group_code,model\nADMS,mad", own) + h2 <- .lnk_config_hash(cfg) + expect_false(identical(h1, h2)) +}) + test_that(".lnk_config_hash does not depend on the host's collation locale", { # Byte order, not LC_COLLATE: en_US and C sort `user_barriers_definite.csv` # and `user_barriers_definite_control.csv` in opposite orders. diff --git a/tests/testthat/test-lnk_preflight_vintage.R b/tests/testthat/test-lnk_preflight_vintage.R index a17720e5..079659ec 100644 --- a/tests/testthat/test-lnk_preflight_vintage.R +++ b/tests/testthat/test-lnk_preflight_vintage.R @@ -11,7 +11,7 @@ fresh_ages <- function() stats::setNames(rep(2, length(all_four)), all_four) test_that(".lnk_vintage_primitives selects the snapshot-loaded tables only", { - # The seven FWA tables are bulk-restored and never ANALYZEd, so including + # The FWA tables are bulk-restored and never ANALYZEd, so including # them would fail every host forever on data that is not the staleness risk. expect_setequal(all_four, c("bcfishobs.observations", "whse_fish.pscis_assessment_svw", From 8a547e276409483b0cd5a5a6f3ec9566fb56e60b Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:20:06 -0700 Subject: [PATCH 4/9] Pass the bundle's cw/mad table to frs_habitat_classify (#286) lnk_pipeline_classify() resolves method_csv (argument, then the bundle's files, then fresh's copy) and hands it to fresh as params_method. Prepare joins mad_m3s from fwa_stream_networks_discharge onto the working streams only; the persisted shape is unchanged. A mad group skips the stream-order rearing bypass, which bcfishpass applies in its cw branch alone. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- R/lnk_pipeline_classify.R | 28 ++++- R/lnk_pipeline_prepare.R | 9 ++ man/lnk_pipeline_classify.Rd | 14 ++- planning/active/progress.md | 1 + planning/active/task_plan.md | 8 +- tests/testthat/test-lnk_pipeline_classify.R | 119 ++++++++++++++++++++ tests/testthat/test-lnk_pipeline_prepare.R | 27 +++++ 7 files changed, 199 insertions(+), 7 deletions(-) diff --git a/R/lnk_pipeline_classify.R b/R/lnk_pipeline_classify.R index d571003e..0e71bb1a 100644 --- a/R/lnk_pipeline_classify.R +++ b/R/lnk_pipeline_classify.R @@ -34,6 +34,16 @@ #' `NULL` uses the config's own `files$parameters_habitat_thresholds`, #' falling back (with a message) to the copy shipped with fresh when #' the config declares none. +#' @param method_csv Path to the per-watershed-group habitat model table +#' (`watershed_group_code`, `model` = `"cw"` or `"mad"`), passed to +#' [fresh::frs_habitat_classify()] as `params_method`. Default `NULL` uses +#' the config's own `files$parameters_habitat_method`, falling back (with a +#' message) to fresh's all-`cw` copy when the config declares none. A group +#' the table does not list classifies on channel width. A `mad` group +#' classifies on `mad_m3s`, which the prepare phase joins onto the working +#' streams table, and skips the stream-order rearing bypass. Read from the +#' path, not from `loaded$parameters_habitat_method`, as the thresholds are, +#' so editing `loaded` has no effect here. #' #' @return `conn` invisibly, for pipe chaining. #' @@ -58,7 +68,8 @@ #' } lnk_pipeline_classify <- function(conn, aoi, cfg, loaded, schema, species = NULL, - thresholds_csv = NULL) { + thresholds_csv = NULL, + method_csv = NULL) { .lnk_validate_identifier(schema, "schema") if (!is.character(aoi) || length(aoi) != 1L || !nzchar(aoi)) { stop("aoi must be a single non-empty string (watershed group code)", @@ -76,6 +87,13 @@ lnk_pipeline_classify <- function(conn, aoi, cfg, loaded, schema, if (!nzchar(thresholds_csv) || !file.exists(thresholds_csv)) { stop("thresholds_csv not found: ", thresholds_csv, call. = FALSE) } + method_csv <- method_csv %||% .lnk_habitat_method_csv(cfg) + if (!nzchar(method_csv) || !file.exists(method_csv)) { + stop("method_csv not found: ", method_csv, call. = FALSE) + } + params_method <- utils::read.csv(method_csv, stringsAsFactors = FALSE, + colClasses = "character", + na.strings = character(0)) species <- species %||% lnk_pipeline_species(cfg, loaded, aoi) if (length(species) == 0L) { @@ -98,6 +116,7 @@ lnk_pipeline_classify <- function(conn, aoi, cfg, loaded, schema, species = species, params = params, params_fresh = loaded$parameters_fresh, + params_method = params_method, gate = TRUE, label_block = "blocked", barrier_overrides = paste0(schema, ".barrier_overrides"), @@ -142,7 +161,12 @@ lnk_pipeline_classify <- function(conn, aoi, cfg, loaded, schema, # `rear[].channel_width_min_bypass = list(stream_order, stream_order_parent_min)`. # We detect that field's presence on any rear rule and call # `frs_order_child` with the embedded parent_order threshold. - for (sp in species) { + # + # Channel-width model only (#286): the bypass stands in for a width test, + # and bcfp applies it inside its cw branch alone. A `mad` group skips it. + aoi_model <- params_method$model[match(aoi, params_method$watershed_group_code)] + species_bypass <- if (identical(aoi_model, "mad")) character(0) else species + for (sp in species_bypass) { rear_rules <- params[[sp]][["rules"]][["rear"]] bypass <- NULL for (rr in rear_rules) { diff --git a/R/lnk_pipeline_prepare.R b/R/lnk_pipeline_prepare.R index 6cc5d8c6..564d2e8d 100644 --- a/R/lnk_pipeline_prepare.R +++ b/R/lnk_pipeline_prepare.R @@ -691,6 +691,15 @@ lnk_pipeline_prepare <- function(conn, aoi, cfg, loaded, schema, cols = c("channel_width", "channel_width_source"), by = "linear_feature_id") + # Mean annual discharge, for watershed groups a bundle's + # parameters_habitat_method puts on the `mad` model (#286). Joined for + # every group so the column is always there; fresh reads it only for + # `mad` groups. Working table only: the persist shape does not carry it. + fresh::frs_col_join(conn, streams_tbl, + from = "whse_basemapping.fwa_stream_networks_discharge", + cols = "mad_m3s", + by = "linear_feature_id") + fresh::frs_col_join(conn, streams_tbl, from = "whse_basemapping.fwa_stream_networks_order_parent", cols = "stream_order_parent", diff --git a/man/lnk_pipeline_classify.Rd b/man/lnk_pipeline_classify.Rd index 0fc05b0b..46682892 100644 --- a/man/lnk_pipeline_classify.Rd +++ b/man/lnk_pipeline_classify.Rd @@ -11,7 +11,8 @@ lnk_pipeline_classify( loaded, schema, species = NULL, - thresholds_csv = NULL + thresholds_csv = NULL, + method_csv = NULL ) } \arguments{ @@ -37,6 +38,17 @@ with the species present in the AOI (via \code{NULL} uses the config's own \code{files$parameters_habitat_thresholds}, falling back (with a message) to the copy shipped with fresh when the config declares none.} + +\item{method_csv}{Path to the per-watershed-group habitat model table +(\code{watershed_group_code}, \code{model} = \code{"cw"} or \code{"mad"}), passed to +\code{\link[fresh:frs_habitat_classify]{fresh::frs_habitat_classify()}} as \code{params_method}. Default \code{NULL} uses +the config's own \code{files$parameters_habitat_method}, falling back (with a +message) to fresh's all-\code{cw} copy when the config declares none. A group +the table does not list classifies on channel width. A \code{mad} group +classifies on \code{mad_m3s}, which the prepare phase joins onto the working +streams table, and skips the stream-order rearing bypass. Read from the +path, not from \code{loaded$parameters_habitat_method}, as the thresholds are, +so editing \code{loaded} has no effect here.} } \value{ \code{conn} invisibly, for pipe chaining. diff --git a/planning/active/progress.md b/planning/active/progress.md index 8ea9d225..861dd090 100644 --- a/planning/active/progress.md +++ b/planning/active/progress.md @@ -8,3 +8,4 @@ - Next: start Phase 1 - Phase 1: fresh pin -> v0.36.2 (floor 0.35.0); `lnk_preflight_fresh(required_formals=)` asserts `frs_habitat_classify(params_method)`; vignette script off `frs_db_conn()` (fresh 0.36.0 precedence flip). Code-check round 1 clean; it flagged stale text (validator/test comments citing v0.33.0, CLAUDE.md `frs_db_conn` line, a skip message), fixed in the same commit - Phase 2: method CSV in the four base bundles (frozen, not csv-synced: B1 from the plan review and code-check round 2, found by both independently), resolver + config_hash fallback, dictionary + tests, discharge in `log_input` primitives, stale `*_mad_*` docs fixed, verify-clean loop over every bundle. Code-check rounds 2 (B1) and 3 (clean) +- Phase 3: `mad_m3s` joined onto working streams; `lnk_pipeline_classify(method_csv=)` passes `params_method`; the stream-order rearing bypass is skipped for a `mad` AOI (G3); tests for capture, fallback, override, the `NA` code, the bypass pair, the join args and the persist shape diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md index 4e76a397..0627acc0 100644 --- a/planning/active/task_plan.md +++ b/planning/active/task_plan.md @@ -44,10 +44,10 @@ needs adding there. - [x] `lnk_config_verify` / audit pass clean on all bundles ## Phase 3: thread through classify + `mad_m3s` on working streams -- [ ] `.lnk_pipeline_prep_network()`: add `fresh::frs_col_join(..., from = "whse_basemapping.fwa_stream_networks_discharge", cols = "mad_m3s", by = "linear_feature_id")` next to the channel_width join. Check that `frs_break_apply` carries it through splits -- [ ] `lnk_pipeline_classify()`: add a `method_csv = NULL` argument, mirroring `thresholds_csv`. Resolve it as `method_csv %||% .lnk_habitat_method_csv(cfg)`, read it, and pass `params_method =` to `frs_habitat_classify()`. Update the roxygen -- [ ] Tests in `test-lnk_pipeline_classify.R`, following the #282 capture pattern with mocked `frs_habitat_classify`: the bundle's table is passed, the undeclared fallback uses fresh's copy and says so, and an explicit `method_csv` wins -- [ ] `lnk_pipeline_connect` stays as it is. `.frs_run_connectivity` is not method-aware (clustering is on gradient), so record that in findings rather than change it +- [x] `.lnk_pipeline_prep_network()`: add `fresh::frs_col_join(..., from = "whse_basemapping.fwa_stream_networks_discharge", cols = "mad_m3s", by = "linear_feature_id")` next to the channel_width join. Check that `frs_break_apply` carries it through splits +- [x] `lnk_pipeline_classify()`: add a `method_csv = NULL` argument, mirroring `thresholds_csv`. Resolve it as `method_csv %||% .lnk_habitat_method_csv(cfg)`, read it, and pass `params_method =` to `frs_habitat_classify()`. Update the roxygen +- [x] Tests in `test-lnk_pipeline_classify.R`, following the #282 capture pattern with mocked `frs_habitat_classify`: the bundle's table is passed, the undeclared fallback uses fresh's copy and says so, and an explicit `method_csv` wins +- [x] `lnk_pipeline_connect` stays as it is. `.frs_run_connectivity` is not method-aware (clustering is on gradient), so record that in findings rather than change it ## Phase 4: live verification (local docker fwapg, scratch schema) - [ ] State the run decisions (config, scratch schema, WSGs, species) before launching diff --git a/tests/testthat/test-lnk_pipeline_classify.R b/tests/testthat/test-lnk_pipeline_classify.R index 9415cf82..1e2b2bcb 100644 --- a/tests/testthat/test-lnk_pipeline_classify.R +++ b/tests/testthat/test-lnk_pipeline_classify.R @@ -104,3 +104,122 @@ test_that("lnk_pipeline_classify: an explicit thresholds_csv wins over the bundl capture_th_csv_classify(cfg, thresholds_csv = explicit_csv), explicit_csv) }) + +# -- method table resolution (#286) ------------------------------------------ + +# Stops at frs_habitat_classify and returns the params_method it was handed. +capture_method_classify <- function(cfg, ...) { + local_mocked_bindings( + .lnk_pipeline_classify_build_breaks = function(...) invisible(NULL) + ) + got <- NULL + local_mocked_bindings( + frs_params = function(...) list(), + frs_habitat_classify = function(..., params_method) { + got <<- params_method + stop("captured", call. = FALSE) + }, + .package = "fresh" + ) + expect_error( + lnk_pipeline_classify("mock", aoi = "BULK", cfg = cfg, loaded = list(), + schema = "w_bulk", species = "BT", ...), + "captured") + got +} + +test_that("lnk_pipeline_classify passes the bundle's method table to fresh", { + csv <- withr::local_tempfile(fileext = ".csv") + writeLines("watershed_group_code,model\nBULK,mad\nADMS,cw", csv) + cfg <- lnk_config("default") + cfg$files$parameters_habitat_method <- list(path = csv) + pm <- capture_method_classify(cfg) + expect_identical(pm$watershed_group_code, c("BULK", "ADMS")) + expect_identical(pm$model, c("mad", "cw")) +}) + +test_that("lnk_pipeline_classify passes a shipped bundle's own method table", { + cfg <- lnk_config("default") + pm <- capture_method_classify(cfg) + shipped <- utils::read.csv(cfg$files$parameters_habitat_method$path, + colClasses = "character") + expect_identical(pm$watershed_group_code, shipped$watershed_group_code) + expect_true(all(pm$model == "cw")) +}) + +test_that("lnk_pipeline_classify falls back to fresh's method table when undeclared", { + skip_if_not_installed("fresh") + cfg <- lnk_config("default") + cfg$files$parameters_habitat_method <- NULL + expect_message(pm <- capture_method_classify(cfg), "fresh's copy") + fresh_pm <- utils::read.csv(system.file("extdata", + "parameters_habitat_method.csv", package = "fresh"), + colClasses = "character") + expect_identical(pm$watershed_group_code, fresh_pm$watershed_group_code) +}) + +test_that("lnk_pipeline_classify: an explicit method_csv wins over the bundle's", { + bundle_csv <- withr::local_tempfile(fileext = ".csv") + explicit_csv <- withr::local_tempfile(fileext = ".csv") + writeLines("watershed_group_code,model\nBULK,cw", bundle_csv) + writeLines("watershed_group_code,model\nBULK,mad", explicit_csv) + cfg <- lnk_config("default") + cfg$files$parameters_habitat_method <- list(path = bundle_csv) + pm <- capture_method_classify(cfg, method_csv = explicit_csv) + expect_identical(pm$model, "mad") +}) + +test_that("lnk_pipeline_classify stops on a missing method_csv", { + cfg <- lnk_config("default") + expect_error( + lnk_pipeline_classify("mock", aoi = "BULK", cfg = cfg, loaded = list(), + schema = "w_bulk", species = "BT", + method_csv = "/no/such/method.csv"), + "method_csv not found") +}) + +test_that("a watershed group coded NA is a code, not a missing value", { + # read.csv's default na.strings would turn a group code "NA" into NA, and + # fresh's match() would then drop it to cw silently. + csv <- withr::local_tempfile(fileext = ".csv") + writeLines("watershed_group_code,model\nNA,mad", csv) + cfg <- lnk_config("default") + pm <- capture_method_classify(cfg, method_csv = csv) + expect_identical(pm$watershed_group_code, "NA") +}) + +# -- stream-order bypass is cw-only (#286) ----------------------------------- + +# Runs classify end to end with fresh mocked; returns the species +# frs_order_child was called for. +bypass_calls <- function(model) { + csv <- withr::local_tempfile(fileext = ".csv") + writeLines(c("watershed_group_code,model", paste0("BULK,", model)), csv) + local_mocked_bindings( + .lnk_pipeline_classify_build_breaks = function(...) invisible(NULL) + ) + called <- character(0) + local_mocked_bindings( + frs_params = function(...) list(BT = list(rules = list(rear = list( + list(channel_width_min_bypass = list(stream_order_parent_min = 5L)))))), + frs_habitat_classify = function(...) invisible(NULL), + frs_order_child = function(conn, ..., species) { + called <<- c(called, species) + invisible(NULL) + }, + .package = "fresh" + ) + lnk_pipeline_classify("mock", aoi = "BULK", cfg = lnk_config("default"), + loaded = list(), schema = "w_bulk", species = "BT", + method_csv = csv) + called +} + +test_that("the stream-order rearing bypass runs for a cw group", { + expect_identical(bypass_calls("cw"), "BT") +}) + +test_that("the stream-order rearing bypass is skipped for a mad group", { + # bcfp applies it inside its channel-width branch only. + expect_identical(bypass_calls("mad"), character(0)) +}) diff --git a/tests/testthat/test-lnk_pipeline_prepare.R b/tests/testthat/test-lnk_pipeline_prepare.R index e2103e8c..77882580 100644 --- a/tests/testthat/test-lnk_pipeline_prepare.R +++ b/tests/testthat/test-lnk_pipeline_prepare.R @@ -535,6 +535,33 @@ test_that(".lnk_pipeline_prep_network loads fresh.streams with FWA filters", { expect_match(joined, "ADD COLUMN id_segment integer") }) +test_that(".lnk_pipeline_prep_network joins mad_m3s onto the working streams (#286)", { + joins <- list() + local_mocked_bindings(.lnk_db_execute = function(conn, sql) invisible(NULL)) + local_mocked_bindings( + frs_col_join = function(conn, table, from, cols, by, ...) { + joins[[length(joins) + 1L]] <<- list(table = table, from = from, + cols = cols, by = by) + invisible(NULL) + }, + frs_col_generate = function(...) invisible(NULL), + .package = "fresh" + ) + .lnk_pipeline_prep_network("mock-conn", aoi = "BULK", schema = "w_bulk") + mad <- Filter(function(j) identical(j$cols, "mad_m3s"), joins) + expect_length(mad, 1L) + expect_identical(mad[[1]]$table, "w_bulk.streams") + expect_identical(mad[[1]]$from, + "whse_basemapping.fwa_stream_networks_discharge") + expect_identical(mad[[1]]$by, "linear_feature_id") +}) + +test_that("mad_m3s stays out of the persisted streams shape (#286)", { + # Decided in #286: working table only, so existing persist schemas need + # no migration. + expect_false("mad_m3s" %in% names(cols_streams)) +}) + # -- prep_overrides control pass-through (manifest-driven) ------------------- test_that(".lnk_pipeline_prep_overrides passes control when manifest declares it", { From 6cf3cbb291d18593c0a805043f05d6459350a219 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:27:27 -0700 Subject: [PATCH 5/9] Record live verification of the cw/mad table (#286) ADMS and BULK classify byte-identically on branch and main, on fresh 0.33.0 and 0.36.2, and with or without mad_m3s on the working streams. Putting ADMS on mad keeps every stream segment inside its MAD range and removes all BT/RB stream habitat; BULK, which has no discharge, loses all stream habitat without an error. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- .../20261002_adms_A0_main_fresh0330.txt | 63 +++++++++ .../20261002_adms_A_main_fresh0362.txt | 63 +++++++++ .../20261002_adms_Anocol_main_fresh0362.txt | 63 +++++++++ .../20261002_adms_B2_branch.txt | 63 +++++++++ .../20261002_adms_B_branch.txt | 63 +++++++++ .../20261002_adms_branch.txt | 118 ++++++++++++++++ .../20261002_adms_cw_check.txt | 52 +++++++ .../params_method_286/20261002_adms_mad.txt | 52 +++++++ .../20261002_adms_rejoin.txt | 2 + .../20261002_bulk_A0_main_fresh0330.txt | 75 ++++++++++ .../20261002_bulk_A_main_fresh0362.txt | 75 ++++++++++ .../20261002_bulk_B_branch.txt | 75 ++++++++++ .../20261002_bulk_branch.txt | 132 ++++++++++++++++++ .../20261002_bulk_cw_check.txt | 62 ++++++++ .../params_method_286/20261002_bulk_mad.txt | 62 ++++++++ data-raw/logs/params_method_286/README.md | 53 +++++++ data-raw/logs/params_method_286/mad_check.R | 51 +++++++ data-raw/logs/params_method_286/method_cw.csv | 3 + .../logs/params_method_286/method_mad.csv | 3 + data-raw/logs/params_method_286/reclassify.R | 34 +++++ .../logs/params_method_286/verify_classify.R | 41 ++++++ planning/active/findings.md | 16 +++ planning/active/progress.md | 1 + planning/active/task_plan.md | 10 +- 24 files changed, 1227 insertions(+), 5 deletions(-) create mode 100644 data-raw/logs/params_method_286/20261002_adms_A0_main_fresh0330.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_A_main_fresh0362.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_Anocol_main_fresh0362.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_B2_branch.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_B_branch.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_branch.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_cw_check.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_mad.txt create mode 100644 data-raw/logs/params_method_286/20261002_adms_rejoin.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_A0_main_fresh0330.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_A_main_fresh0362.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_B_branch.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_branch.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_cw_check.txt create mode 100644 data-raw/logs/params_method_286/20261002_bulk_mad.txt create mode 100644 data-raw/logs/params_method_286/README.md create mode 100644 data-raw/logs/params_method_286/mad_check.R create mode 100644 data-raw/logs/params_method_286/method_cw.csv create mode 100644 data-raw/logs/params_method_286/method_mad.csv create mode 100644 data-raw/logs/params_method_286/reclassify.R create mode 100644 data-raw/logs/params_method_286/verify_classify.R diff --git a/data-raw/logs/params_method_286/20261002_adms_A0_main_fresh0330.txt b/data-raw/logs/params_method_286/20261002_adms_A0_main_fresh0330.txt new file mode 100644 index 00000000..58ffa670 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_A0_main_fresh0330.txt @@ -0,0 +1,63 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## reclassify ADMS default zz286_adms repo=wt_main method_csv= link=0.54.0 fresh=0.33.0 + species_code n n_spawn n_rear spawn_km rear_km + BT 39423 1484 2393 364.8 531.6 + CH 39423 1068 1470 278.2 353.3 + CO 39423 1238 1516 314.0 362.9 + RB 39423 1242 2006 307.0 439.3 + SK 39423 912 360 218.2 229.9 + digest + 18f2d027cf834ee330995ec62b71dd3b + ac5e427b0e480d50e99cd8404df65157 + 1a9669255ed036a8050187a82e986176 + 6662d822c956aeab9575b11f2ff791d0 + 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_A_main_fresh0362.txt b/data-raw/logs/params_method_286/20261002_adms_A_main_fresh0362.txt new file mode 100644 index 00000000..858ffecc --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_A_main_fresh0362.txt @@ -0,0 +1,63 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## reclassify ADMS default zz286_adms repo=wt_main method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 39423 1484 2393 364.8 531.6 + CH 39423 1068 1470 278.2 353.3 + CO 39423 1238 1516 314.0 362.9 + RB 39423 1242 2006 307.0 439.3 + SK 39423 912 360 218.2 229.9 + digest + 18f2d027cf834ee330995ec62b71dd3b + ac5e427b0e480d50e99cd8404df65157 + 1a9669255ed036a8050187a82e986176 + 6662d822c956aeab9575b11f2ff791d0 + 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_Anocol_main_fresh0362.txt b/data-raw/logs/params_method_286/20261002_adms_Anocol_main_fresh0362.txt new file mode 100644 index 00000000..858ffecc --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_Anocol_main_fresh0362.txt @@ -0,0 +1,63 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## reclassify ADMS default zz286_adms repo=wt_main method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 39423 1484 2393 364.8 531.6 + CH 39423 1068 1470 278.2 353.3 + CO 39423 1238 1516 314.0 362.9 + RB 39423 1242 2006 307.0 439.3 + SK 39423 912 360 218.2 229.9 + digest + 18f2d027cf834ee330995ec62b71dd3b + ac5e427b0e480d50e99cd8404df65157 + 1a9669255ed036a8050187a82e986176 + 6662d822c956aeab9575b11f2ff791d0 + 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_B2_branch.txt b/data-raw/logs/params_method_286/20261002_adms_B2_branch.txt new file mode 100644 index 00000000..1263979d --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_B2_branch.txt @@ -0,0 +1,63 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## reclassify ADMS default zz286_adms repo=br method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 39423 1484 2393 364.8 531.6 + CH 39423 1068 1470 278.2 353.3 + CO 39423 1238 1516 314.0 362.9 + RB 39423 1242 2006 307.0 439.3 + SK 39423 912 360 218.2 229.9 + digest + 18f2d027cf834ee330995ec62b71dd3b + ac5e427b0e480d50e99cd8404df65157 + 1a9669255ed036a8050187a82e986176 + 6662d822c956aeab9575b11f2ff791d0 + 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_B_branch.txt b/data-raw/logs/params_method_286/20261002_adms_B_branch.txt new file mode 100644 index 00000000..1263979d --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_B_branch.txt @@ -0,0 +1,63 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## reclassify ADMS default zz286_adms repo=br method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 39423 1484 2393 364.8 531.6 + CH 39423 1068 1470 278.2 353.3 + CO 39423 1238 1516 314.0 362.9 + RB 39423 1242 2006 307.0 439.3 + SK 39423 912 360 218.2 229.9 + digest + 18f2d027cf834ee330995ec62b71dd3b + ac5e427b0e480d50e99cd8404df65157 + 1a9669255ed036a8050187a82e986176 + 6662d822c956aeab9575b11f2ff791d0 + 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_branch.txt b/data-raw/logs/params_method_286/20261002_adms_branch.txt new file mode 100644 index 00000000..b17c53a5 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_branch.txt @@ -0,0 +1,118 @@ +NOTICE: schema "zz286_adms" does not exist, skipping + +NOTICE: schema "fresh" already exists, skipping + +NOTICE: table "barriers_definite" does not exist, skipping + +NOTICE: table "observations" does not exist, skipping + +NOTICE: table "streams_blk" does not exist, skipping + +NOTICE: table "gradient_barriers_raw" does not exist, skipping + +NOTICE: table "natural_barriers" does not exist, skipping + +NOTICE: table "barrier_overrides" does not exist, skipping + +NOTICE: table "barriers_bt" does not exist, skipping + +NOTICE: table "barriers_ch" does not exist, skipping + +NOTICE: table "barriers_cm" does not exist, skipping + +NOTICE: table "barriers_co" does not exist, skipping + +NOTICE: table "barriers_pk" does not exist, skipping + +NOTICE: table "barriers_sk" does not exist, skipping + +NOTICE: table "barriers_st" does not exist, skipping + +NOTICE: table "barriers_wct" does not exist, skipping + +NOTICE: table "barriers_ko" does not exist, skipping + +NOTICE: table "barriers_rb" does not exist, skipping + +NOTICE: table "barriers_gr" does not exist, skipping + +NOTICE: table "gradient_barriers_minimal" does not exist, skipping + +NOTICE: table "streams" does not exist, skipping + +NOTICE: table "streams_habitat" does not exist, skipping + +NOTICE: table "cabd_exclusions" does not exist, skipping + +NOTICE: table "cabd_blkey_xref" does not exist, skipping + +NOTICE: table "cabd_passability_status_updates" does not exist, skipping + +NOTICE: table "cabd_additions" does not exist, skipping + +NOTICE: table "cabd_dams_raw" does not exist, skipping + +NOTICE: table "dams" does not exist, skipping + +NOTICE: table "pscis_modelledcrossings_streams_xref" does not exist, skipping + +NOTICE: table "pscis_stream_candidates" does not exist, skipping + +NOTICE: table "pscis_streams_150m" does not exist, skipping + +NOTICE: table "pscis_picked" does not exist, skipping + +NOTICE: table "pscis" does not exist, skipping + +NOTICE: table "crossings_lookup" does not exist, skipping + +NOTICE: table "barriers_anthropogenic" does not exist, skipping + +NOTICE: table "barriers_pscis" does not exist, skipping + +NOTICE: table "barriers_dams" does not exist, skipping + +NOTICE: table "barriers_remediations" does not exist, skipping + +NOTICE: table "observations_breaks" does not exist, skipping + +NOTICE: table "habitat_endpoints" does not exist, skipping + +NOTICE: table "crossings_breaks" does not exist, skipping + +NOTICE: table "streams_breaks" does not exist, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "frs_clusters_bt" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +## ADMS default zz286_adms link=0.54.0 fresh=0.36.2 cfg_hash=sha256:576b4c168da028e839d4f5927960e24c5ff683f6d0eea91536389d49bedb60e1 1.7 min +streams n=39423 digest=bbed26e41a4206c860a51af02e27ecc7 + species_code n n_spawn n_rear digest + BT 39423 1484 2393 18f2d027cf834ee330995ec62b71dd3b + CH 39423 1068 1470 ac5e427b0e480d50e99cd8404df65157 + CO 39423 1238 1516 1a9669255ed036a8050187a82e986176 + RB 39423 1242 2006 6662d822c956aeab9575b11f2ff791d0 + SK 39423 912 360 3074c3b02d2b93fbbdb193e3da28c26e diff --git a/data-raw/logs/params_method_286/20261002_adms_cw_check.txt b/data-raw/logs/params_method_286/20261002_adms_cw_check.txt new file mode 100644 index 00000000..fe5e6c82 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_cw_check.txt @@ -0,0 +1,52 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +## mad_check ADMS zz286_adms method_csv=method_cw.csv link=0.54.0 fresh=0.36.2 (classify only, overlay off) + species_code spawn_km rear_km spawn_km_nowb rear_km_nowb lake_km wetland_km + BT 364.8 560.8 214.5 346.2 151.6 36.9 + CH 278.1 368.0 128.3 169.8 137.8 29.9 + CO 312.3 368.0 162.1 169.8 144.4 30.5 + RB 307.0 439.3 172.7 251.9 147.9 29.0 + SK 234.3 229.9 90.5 0.0 134.9 0.0 + spawn_null_mad rear_null_mad spawn_out spawn_no_th rear_out rear_no_th + 1 2 0 1484 0 2621 + 1 1 157 0 487 0 + 1 1 192 0 305 0 + 1 1 0 1242 0 2006 + 1 2 127 0 0 360 +[1] 0 diff --git a/data-raw/logs/params_method_286/20261002_adms_mad.txt b/data-raw/logs/params_method_286/20261002_adms_mad.txt new file mode 100644 index 00000000..0f93a3bd --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_mad.txt @@ -0,0 +1,52 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +## mad_check ADMS zz286_adms method_csv=method_mad.csv link=0.54.0 fresh=0.36.2 (classify only, overlay off) + species_code spawn_km rear_km spawn_km_nowb rear_km_nowb lake_km wetland_km + BT 0.0 63.5 0.0 0.0 252.4 65.8 + CH 256.3 330.7 106.4 132.4 83.2 15.4 + CO 290.7 331.2 140.1 172.3 80.8 33.2 + RB 0.0 52.9 0.0 0.0 246.7 55.7 + SK 226.2 229.9 81.8 0.0 229.9 0.0 + spawn_null_mad rear_null_mad spawn_out spawn_no_th rear_out rear_no_th + 0 0 0 0 0 474 + 0 0 0 0 222 0 + 0 0 0 0 128 0 + 0 0 0 0 0 387 + 0 2 0 0 0 360 +[1] 0 diff --git a/data-raw/logs/params_method_286/20261002_adms_rejoin.txt b/data-raw/logs/params_method_286/20261002_adms_rejoin.txt new file mode 100644 index 00000000..0c91bbeb --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_adms_rejoin.txt @@ -0,0 +1,2 @@ + count +1 39414 diff --git a/data-raw/logs/params_method_286/20261002_bulk_A0_main_fresh0330.txt b/data-raw/logs/params_method_286/20261002_bulk_A0_main_fresh0330.txt new file mode 100644 index 00000000..025dbe64 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_A0_main_fresh0330.txt @@ -0,0 +1,75 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +## reclassify BULK default zz286_bulk repo=wt_main method_csv= link=0.54.0 fresh=0.33.0 + species_code n n_spawn n_rear spawn_km rear_km + BT 87638 5553 9695 1853.4 3010.3 + CH 87638 3854 6701 1264.1 2038.2 + CO 87638 5375 7548 1813.7 2295.7 + PK 87638 5447 0 1864.2 0.0 + RB 87638 5445 10070 1818.7 3013.3 + SK 87638 224 139 81.4 64.6 + ST 87638 3955 8124 1295.5 2552.1 + digest + 12070f6142acfe491c041e7d42d2ace0 + 13900c2faa6d5ec1564feb18e7e554c7 + 23a1b679a66b4898efe102cdd6b8b020 + 0755c74eb7ea6636a357a4ba26e2eba3 + 11e053f8970b99879991deb384c35ebd + 63d3d58098baf8cae37e26fe573612c7 + 74468447d499c3409bef399182451150 diff --git a/data-raw/logs/params_method_286/20261002_bulk_A_main_fresh0362.txt b/data-raw/logs/params_method_286/20261002_bulk_A_main_fresh0362.txt new file mode 100644 index 00000000..72533228 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_A_main_fresh0362.txt @@ -0,0 +1,75 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +## reclassify BULK default zz286_bulk repo=wt_main method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 87638 5553 9695 1853.4 3010.3 + CH 87638 3854 6701 1264.1 2038.2 + CO 87638 5375 7548 1813.7 2295.7 + PK 87638 5447 0 1864.2 0.0 + RB 87638 5445 10070 1818.7 3013.3 + SK 87638 224 139 81.4 64.6 + ST 87638 3955 8124 1295.5 2552.1 + digest + 12070f6142acfe491c041e7d42d2ace0 + 13900c2faa6d5ec1564feb18e7e554c7 + 23a1b679a66b4898efe102cdd6b8b020 + 0755c74eb7ea6636a357a4ba26e2eba3 + 11e053f8970b99879991deb384c35ebd + 63d3d58098baf8cae37e26fe573612c7 + 74468447d499c3409bef399182451150 diff --git a/data-raw/logs/params_method_286/20261002_bulk_B_branch.txt b/data-raw/logs/params_method_286/20261002_bulk_B_branch.txt new file mode 100644 index 00000000..3c2d9651 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_B_branch.txt @@ -0,0 +1,75 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +NOTICE: schema "pg_temp" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +## reclassify BULK default zz286_bulk repo=br method_csv= link=0.54.0 fresh=0.36.2 + species_code n n_spawn n_rear spawn_km rear_km + BT 87638 5553 9695 1853.4 3010.3 + CH 87638 3854 6701 1264.1 2038.2 + CO 87638 5375 7548 1813.7 2295.7 + PK 87638 5447 0 1864.2 0.0 + RB 87638 5445 10070 1818.7 3013.3 + SK 87638 224 139 81.4 64.6 + ST 87638 3955 8124 1295.5 2552.1 + digest + 12070f6142acfe491c041e7d42d2ace0 + 13900c2faa6d5ec1564feb18e7e554c7 + 23a1b679a66b4898efe102cdd6b8b020 + 0755c74eb7ea6636a357a4ba26e2eba3 + 11e053f8970b99879991deb384c35ebd + 63d3d58098baf8cae37e26fe573612c7 + 74468447d499c3409bef399182451150 diff --git a/data-raw/logs/params_method_286/20261002_bulk_branch.txt b/data-raw/logs/params_method_286/20261002_bulk_branch.txt new file mode 100644 index 00000000..6b072e29 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_branch.txt @@ -0,0 +1,132 @@ +NOTICE: schema "zz286_bulk" does not exist, skipping + +NOTICE: schema "fresh" already exists, skipping + +Override validation: zz286_bulk.pscis_fixes vs zz286_bulk.crossings + Total overrides: 589 + Valid (matched): 521 + Orphans: 68 <-- not found in crossings + Duplicates: 0 +Updated 521 of 5568 rows (barrier_status) +NOTICE: table "observations" does not exist, skipping + +NOTICE: table "streams_blk" does not exist, skipping + +NOTICE: table "gradient_barriers_raw" does not exist, skipping + +NOTICE: table "natural_barriers" does not exist, skipping + +NOTICE: table "barrier_overrides" does not exist, skipping + +NOTICE: table "barriers_bt" does not exist, skipping + +NOTICE: table "barriers_ch" does not exist, skipping + +NOTICE: table "barriers_cm" does not exist, skipping + +NOTICE: table "barriers_co" does not exist, skipping + +NOTICE: table "barriers_pk" does not exist, skipping + +NOTICE: table "barriers_sk" does not exist, skipping + +NOTICE: table "barriers_st" does not exist, skipping + +NOTICE: table "barriers_wct" does not exist, skipping + +NOTICE: table "barriers_ko" does not exist, skipping + +NOTICE: table "barriers_rb" does not exist, skipping + +NOTICE: table "barriers_gr" does not exist, skipping + +NOTICE: table "gradient_barriers_minimal" does not exist, skipping + +NOTICE: table "streams" does not exist, skipping + +NOTICE: table "streams_habitat" does not exist, skipping + +NOTICE: table "cabd_exclusions" does not exist, skipping + +NOTICE: table "cabd_blkey_xref" does not exist, skipping + +NOTICE: table "cabd_passability_status_updates" does not exist, skipping + +NOTICE: table "cabd_additions" does not exist, skipping + +NOTICE: table "cabd_dams_raw" does not exist, skipping + +NOTICE: table "dams" does not exist, skipping + +NOTICE: table "pscis_modelledcrossings_streams_xref" does not exist, skipping + +NOTICE: table "pscis_stream_candidates" does not exist, skipping + +NOTICE: table "pscis_streams_150m" does not exist, skipping + +NOTICE: table "pscis_picked" does not exist, skipping + +NOTICE: table "pscis" does not exist, skipping + +NOTICE: table "crossings_lookup" does not exist, skipping + +NOTICE: table "barriers_anthropogenic" does not exist, skipping + +NOTICE: table "barriers_pscis" does not exist, skipping + +NOTICE: table "barriers_dams" does not exist, skipping + +NOTICE: table "barriers_remediations" does not exist, skipping + +NOTICE: table "observations_breaks" does not exist, skipping + +NOTICE: table "habitat_endpoints" does not exist, skipping + +NOTICE: table "crossings_breaks" does not exist, skipping + +NOTICE: table "streams_breaks" does not exist, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: table "frs_clusters_bt" does not exist, skipping + +NOTICE: table "frs_clusters_ch" does not exist, skipping + +NOTICE: table "frs_clusters_co" does not exist, skipping + +NOTICE: table "frs_clusters_sk" does not exist, skipping + +NOTICE: table "frs_qual_spawn_sk" does not exist, skipping + +NOTICE: table "frs_trace_lfid_sk" does not exist, skipping + +NOTICE: table "frs_clusters_st" does not exist, skipping + +## BULK default zz286_bulk link=0.54.0 fresh=0.36.2 cfg_hash=sha256:576b4c168da028e839d4f5927960e24c5ff683f6d0eea91536389d49bedb60e1 4.4 min +streams n=87638 digest=b5bd1310bcda04534eb3ce7c9869fbc5 + species_code n n_spawn n_rear digest + BT 87638 5553 9695 12070f6142acfe491c041e7d42d2ace0 + CH 87638 3854 6701 13900c2faa6d5ec1564feb18e7e554c7 + CO 87638 5375 7548 23a1b679a66b4898efe102cdd6b8b020 + PK 87638 5447 0 0755c74eb7ea6636a357a4ba26e2eba3 + RB 87638 5445 10070 11e053f8970b99879991deb384c35ebd + SK 87638 224 139 63d3d58098baf8cae37e26fe573612c7 + ST 87638 3955 8124 74468447d499c3409bef399182451150 diff --git a/data-raw/logs/params_method_286/20261002_bulk_cw_check.txt b/data-raw/logs/params_method_286/20261002_bulk_cw_check.txt new file mode 100644 index 00000000..3c8e0470 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_cw_check.txt @@ -0,0 +1,62 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +## mad_check BULK zz286_bulk method_csv=method_cw.csv link=0.54.0 fresh=0.36.2 (classify only, overlay off) + species_code spawn_km rear_km spawn_km_nowb rear_km_nowb lake_km wetland_km + BT 1853.4 3300.5 1403.1 2390.3 77.7 243.4 + CH 1263.8 2398.6 815.0 1557.2 42.5 225.3 + CO 1790.2 2398.6 1340.0 1557.2 85.9 228.5 + PK 1862.0 0.0 1411.2 0.0 0.0 0.0 + RB 1818.7 3013.3 1368.5 2116.9 77.2 234.5 + SK 1111.3 64.6 669.9 0.0 32.1 0.0 + ST 1291.2 2990.5 842.3 2101.9 58.4 235.5 + spawn_null_mad rear_null_mad spawn_out spawn_no_th rear_out rear_no_th + 5553 10981 0 5553 0 10981 + 3850 8019 0 0 0 0 + 5257 8019 0 0 0 0 + 5435 0 0 0 0 0 + 5445 10070 0 5445 0 10070 + 3583 139 0 0 0 139 + 3922 9863 0 0 0 0 +[1] 0 diff --git a/data-raw/logs/params_method_286/20261002_bulk_mad.txt b/data-raw/logs/params_method_286/20261002_bulk_mad.txt new file mode 100644 index 00000000..2ba05ba3 --- /dev/null +++ b/data-raw/logs/params_method_286/20261002_bulk_mad.txt @@ -0,0 +1,62 @@ +NOTICE: relation "streams_blk_idx" already exists, skipping + +NOTICE: relation "streams_blk_drm_idx" already exists, skipping + +NOTICE: relation "streams_wscode_gist_idx" already exists, skipping + +NOTICE: relation "streams_wscode_btree_idx" already exists, skipping + +NOTICE: relation "streams_localcode_gist_idx" already exists, skipping + +NOTICE: relation "streams_localcode_btree_idx" already exists, skipping + +NOTICE: relation "streams_lfid_idx" already exists, skipping + +NOTICE: relation "streams_wsg_idx" already exists, skipping + +NOTICE: relation "streams_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat" already exists, skipping + +NOTICE: table "streams_acc_015" does not exist, skipping + +NOTICE: table "streams_acc_02" does not exist, skipping + +NOTICE: table "streams_acc_025" does not exist, skipping + +NOTICE: table "streams_acc_025_ovr_bt" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_ch" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_co" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_pk" does not exist, skipping + +NOTICE: table "streams_acc_015_ovr_sk" does not exist, skipping + +NOTICE: table "streams_acc_02_ovr_st" does not exist, skipping + +NOTICE: relation "streams_habitat_wsg_idx" already exists, skipping + +NOTICE: relation "streams_habitat_id_segment_idx" already exists, skipping + +NOTICE: relation "streams_habitat_species_code_idx" already exists, skipping + +## mad_check BULK zz286_bulk method_csv=method_mad.csv link=0.54.0 fresh=0.36.2 (classify only, overlay off) + species_code spawn_km rear_km spawn_km_nowb rear_km_nowb lake_km wetland_km + BT 0 458.8 0 0 158.7 522.9 + CH 0 391.3 0 0 0.0 0.0 + CO 0 391.3 0 0 0.0 0.0 + PK 0 0.0 0 0 0.0 0.0 + RB 0 445.2 0 0 157.5 508.8 + SK 0 64.6 0 0 64.6 0.0 + ST 0 437.3 0 0 0.0 0.0 + spawn_null_mad rear_null_mad spawn_out spawn_no_th rear_out rear_no_th + 0 2690 0 0 0 2690 + 0 2187 0 0 0 0 + 0 2187 0 0 0 0 + 0 0 0 0 0 0 + 0 2609 0 0 0 2609 + 0 139 0 0 0 139 + 0 2545 0 0 0 0 +[1] 0 diff --git a/data-raw/logs/params_method_286/README.md b/data-raw/logs/params_method_286/README.md new file mode 100644 index 00000000..622d8b23 --- /dev/null +++ b/data-raw/logs/params_method_286/README.md @@ -0,0 +1,53 @@ +# #286 live verification — per-WSG cw/mad method table + +Local docker fwapg (:5432), 2026-10-02 UTC. Config `default`. Scratch working schemas +`zz286_adms` and `zz286_bulk` only; nothing persisted to `fresh` / `fresh_default`. +Branch code ran from a frozen copy of the working tree, main from a worktree at +`f462113`. fresh 0.36.2 (`e3a37f0`), plus fresh 0.33.0 (`7f12d99`) in a separate library +for the A0 runs (`LNK_LIBPRE`). + +- `verify_classify.R` runs setup → connect on one WSG and digests `streams_habitat` per + species (copied from `../habitat_thresholds_282/`). +- `reclassify.R` re-runs classify + connect on a prepared schema, so segmentation is held + fixed. It optionally takes `method_csv`. +- `mad_check.R` runs classify only, with the overlay off and no connect, under a given + method table, then checks fresh's mad invariants. The `*_nowb` columns are segments + outside any waterbody (`waterbody_key IS NULL`). + +## No-change proof (per-species `streams_habitat` digests, after connect) + +| WSG | runs compared | result | +|---|---|---| +| ADMS | full branch run; B branch; A main + fresh 0.36.2; A0 main + fresh 0.33.0; A_nocol main with `mad_m3s` dropped; B2 branch after re-joining it | **all identical**, 5 species | +| BULK | full branch run; B branch; A main + fresh 0.36.2; A0 main + fresh 0.33.0 | **all identical**, 7 species | + +So the fresh 0.33.0 → 0.36.2 bump, the extra `mad_m3s` column and the all-`cw` method +table each move nothing. `mad_m3s` survives breaking: 39,414 of ADMS's 39,423 working +segments carry it. + +## `mad` takes effect (classify only, overlay off) + +ADMS has discharge for 10,449 of 11,520 FWA segments. Stream (non-waterbody) km, +cw → mad: + +| species | spawn cw → mad | rear cw → mad | +|---|---|---| +| BT | 214.5 → 0 | 346.2 → 0 | +| RB | 172.7 → 0 | 251.9 → 0 | +| CH | 128.3 → 106.4 | 169.8 → 132.4 | +| CO | 162.1 → 140.1 | 169.8 → 172.3 | +| SK | 90.5 → 81.8 | 0 → 0 | + +- **Invariants hold.** No stream spawning or rearing segment of CH/CO/SK has `mad_m3s` + outside `[*_mad_min, *_mad_max]` or NULL. +- **BT and RB keep 63.5 and 52.9 km of rearing**, all of it inside waterbodies: the + wetland-polygon rules and the 1050/1150 `thresholds: false` edges. fresh's lake and + wetland rules inherit no thresholds under either model (`.frs_rule_to_sql`), so this + is intended. +- **222 CH and 128 CO rearing segments sit below the MAD minimum.** They come from those + same non-inheriting waterbody rules, not from a broken invariant. + +BULK has **no discharge at all**. Under `mad`, every species has 0 km of stream spawning +and rearing, and nothing errors. The rearing that remains is waterbody rules only (CH +391.3 km, BT 458.8 km). This is why coverage has to be checked before a group is moved +to `mad`. diff --git a/data-raw/logs/params_method_286/mad_check.R b/data-raw/logs/params_method_286/mad_check.R new file mode 100644 index 00000000..cb55f411 --- /dev/null +++ b/data-raw/logs/params_method_286/mad_check.R @@ -0,0 +1,51 @@ +# Usage: Rscript mad_check.R +# Classify ONLY (no overlay, no connect) on a prepared working schema with the +# given method table, then check fresh's mad invariants straight off +# frs_habitat_classify's output: +# - every spawning / rearing segment has mad_m3s inside [*_mad_min, *_mad_max] +# - no segment with NULL mad_m3s is spawning or rearing +# *_nowb: segments outside any waterbody (waterbody_key IS NULL) -- the +# "stream habitat" that species without MAD thresholds must have none of. +# and report stream / lake / wetland km per species. Config `default`. +args <- commandArgs(trailingOnly = TRUE) +repo <- args[1]; aoi <- args[2]; schema <- args[3]; method_csv <- args[4] +suppressMessages(devtools::load_all(repo, quiet = TRUE)) +conn <- lnk_db_conn(dbname = "fwapg", host = "localhost", port = 5432L, + user = "postgres", password = "postgres") +cfg <- lnk_config("default") +cfg$pipeline$apply_habitat_overlay <- FALSE +loaded <- suppressWarnings(lnk_load_overrides(cfg)) +lnk_pipeline_classify(conn, aoi = aoi, cfg = cfg, loaded = loaded, + schema = schema, method_csv = method_csv) +th <- utils::read.csv(.lnk_habitat_thresholds_csv(cfg)) +th <- th[, c("species_code", "spawn_mad_min", "spawn_mad_max", + "rear_mad_min", "rear_mad_max")] +DBI::dbWriteTable(conn, DBI::Id(schema = schema, table = "zz_mad_th"), th, + overwrite = TRUE) +q <- sprintf( + "SELECT h.species_code, + sum(CASE WHEN h.spawning THEN s.length_metre ELSE 0 END)/1000 AS spawn_km, + sum(CASE WHEN h.rearing THEN s.length_metre ELSE 0 END)/1000 AS rear_km, + sum(CASE WHEN h.spawning AND s.waterbody_key IS NULL THEN s.length_metre ELSE 0 END)/1000 AS spawn_km_nowb, + sum(CASE WHEN h.rearing AND s.waterbody_key IS NULL THEN s.length_metre ELSE 0 END)/1000 AS rear_km_nowb, + sum(CASE WHEN h.lake_rearing THEN s.length_metre ELSE 0 END)/1000 AS lake_km, + sum(CASE WHEN h.wetland_rearing THEN s.length_metre ELSE 0 END)/1000 AS wetland_km, + count(*) FILTER (WHERE h.spawning AND s.mad_m3s IS NULL) AS spawn_null_mad, + count(*) FILTER (WHERE h.rearing AND s.mad_m3s IS NULL) AS rear_null_mad, + count(*) FILTER (WHERE h.spawning AND NOT (s.mad_m3s BETWEEN t.spawn_mad_min AND t.spawn_mad_max)) AS spawn_out, + count(*) FILTER (WHERE h.spawning AND t.spawn_mad_min IS NULL) AS spawn_no_th, + count(*) FILTER (WHERE h.rearing AND NOT (s.mad_m3s BETWEEN t.rear_mad_min AND t.rear_mad_max)) AS rear_out, + count(*) FILTER (WHERE h.rearing AND t.rear_mad_min IS NULL) AS rear_no_th + FROM %1$s.streams_habitat h + JOIN %1$s.streams s USING (id_segment) + LEFT JOIN %1$s.zz_mad_th t USING (species_code) + GROUP BY h.species_code ORDER BY h.species_code", schema) +d <- DBI::dbGetQuery(conn, q) +num <- vapply(d, is.numeric, logical(1)) +d[num] <- lapply(d[num], function(x) round(as.numeric(x), 1)) +cat(sprintf("## mad_check %s %s method_csv=%s link=%s fresh=%s (classify only, overlay off)\n", + aoi, schema, basename(method_csv), utils::packageVersion("link"), + utils::packageVersion("fresh"))) +print(d, row.names = FALSE) +DBI::dbExecute(conn, sprintf("DROP TABLE %s.zz_mad_th", schema)) +DBI::dbDisconnect(conn) diff --git a/data-raw/logs/params_method_286/method_cw.csv b/data-raw/logs/params_method_286/method_cw.csv new file mode 100644 index 00000000..b756c6ea --- /dev/null +++ b/data-raw/logs/params_method_286/method_cw.csv @@ -0,0 +1,3 @@ +watershed_group_code,model +ADMS,cw +BULK,cw diff --git a/data-raw/logs/params_method_286/method_mad.csv b/data-raw/logs/params_method_286/method_mad.csv new file mode 100644 index 00000000..1cffc23a --- /dev/null +++ b/data-raw/logs/params_method_286/method_mad.csv @@ -0,0 +1,3 @@ +watershed_group_code,model +ADMS,mad +BULK,mad diff --git a/data-raw/logs/params_method_286/reclassify.R b/data-raw/logs/params_method_286/reclassify.R new file mode 100644 index 00000000..394f392b --- /dev/null +++ b/data-raw/logs/params_method_286/reclassify.R @@ -0,0 +1,34 @@ +# Usage: Rscript reclassify.R [method_csv] +# Re-runs ONLY classify + connect on an already-prepared working schema, so +# segmentation is held fixed and only the classify path is exercised. With a +# 5th argument, passes it as method_csv (needs a repo that has the argument). +args <- commandArgs(trailingOnly = TRUE) +repo <- args[1]; cfg_arg <- args[2]; aoi <- args[3]; schema <- args[4] +method_csv <- if (length(args) >= 5L) args[5] else NULL +# LNK_LIBPRE: a library to put first, e.g. one holding fresh v0.33.0. +if (nzchar(Sys.getenv("LNK_LIBPRE"))) .libPaths(c(Sys.getenv("LNK_LIBPRE"), .libPaths())) +suppressMessages(devtools::load_all(repo, quiet = TRUE)) +conn <- lnk_db_conn(dbname = "fwapg", host = "localhost", port = 5432L, + user = "postgres", password = "postgres") +cfg <- lnk_config(cfg_arg) +loaded <- suppressWarnings(lnk_load_overrides(cfg)) +if (is.null(method_csv)) { + lnk_pipeline_classify(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +} else { + lnk_pipeline_classify(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema, + method_csv = method_csv) +} +lnk_pipeline_connect(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +d <- DBI::dbGetQuery(conn, sprintf( + "SELECT h.species_code, count(*) AS n, + sum(h.spawning::int) AS n_spawn, sum(h.rearing::int) AS n_rear, + round(sum(CASE WHEN h.spawning THEN s.length_metre ELSE 0 END)::numeric / 1000, 1) AS spawn_km, + round(sum(CASE WHEN h.rearing THEN s.length_metre ELSE 0 END)::numeric / 1000, 1) AS rear_km, + md5(string_agg(h::text, '|' ORDER BY h.id_segment, h.species_code)) AS digest + FROM %1$s.streams_habitat h JOIN %1$s.streams s USING (id_segment) + GROUP BY h.species_code ORDER BY h.species_code", schema)) +cat(sprintf("## reclassify %s %s %s repo=%s method_csv=%s link=%s fresh=%s\n", aoi, + cfg$name, schema, basename(repo), method_csv %||% "", + utils::packageVersion("link"), utils::packageVersion("fresh"))) +print(d, row.names = FALSE) +DBI::dbDisconnect(conn) diff --git a/data-raw/logs/params_method_286/verify_classify.R b/data-raw/logs/params_method_286/verify_classify.R new file mode 100644 index 00000000..45513fc4 --- /dev/null +++ b/data-raw/logs/params_method_286/verify_classify.R @@ -0,0 +1,41 @@ +# Usage: Rscript verify_classify.R +# Runs link phases setup..connect into a scratch working schema (never persists), +# then prints a per-species digest of .streams_habitat. +args <- commandArgs(trailingOnly = TRUE) +repo <- args[1]; cfg_arg <- args[2]; aoi <- args[3]; schema <- args[4] +suppressMessages(devtools::load_all(repo, quiet = TRUE)) +conn <- lnk_db_conn(dbname = "fwapg", host = "localhost", port = 5432L, + user = "postgres", password = "postgres") +cfg <- lnk_config(cfg_arg) +loaded <- suppressWarnings(lnk_load_overrides(cfg)) +t0 <- Sys.time() +lnk_pipeline_setup(conn, schema, overwrite = TRUE) +lnk_pipeline_load(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +lnk_pipeline_prepare(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema, + conn_tunnel = conn) +lnk_pipeline_crossings(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +lnk_pipeline_break(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +lnk_pipeline_classify(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +lnk_pipeline_connect(conn, aoi = aoi, cfg = cfg, loaded = loaded, schema = schema) +cols <- DBI::dbGetQuery(conn, sprintf( + "SELECT column_name FROM information_schema.columns + WHERE table_schema = '%s' AND table_name = 'streams_habitat' + ORDER BY column_name", schema))$column_name +keycols <- intersect(c("id_segment", "species_code"), cols) +d <- DBI::dbGetQuery(conn, sprintf( + "SELECT species_code, count(*) AS n, + sum(spawning::int) AS n_spawn, sum(rearing::int) AS n_rear, + md5(string_agg(t::text, '|' ORDER BY %s)) AS digest + FROM %s.streams_habitat t GROUP BY species_code ORDER BY species_code", + paste(keycols, collapse = ", "), schema)) +seg <- DBI::dbGetQuery(conn, sprintf( + "SELECT count(*) AS n, md5(string_agg(id_segment::text || ':' || + round(gradient::numeric, 6)::text, '|' ORDER BY id_segment)) AS digest + FROM %s.streams", schema)) +cat(sprintf("## %s %s %s link=%s fresh=%s cfg_hash=%s %.1f min\n", aoi, + cfg$name, schema, utils::packageVersion("link"), + utils::packageVersion("fresh"), .lnk_config_hash(cfg), + as.numeric(difftime(Sys.time(), t0, units = "mins")))) +cat(sprintf("streams n=%s digest=%s\n", format(seg$n), seg$digest)) +print(d, row.names = FALSE) +DBI::dbDisconnect(conn) diff --git a/planning/active/findings.md b/planning/active/findings.md index d475d6cb..cb41c31f 100644 --- a/planning/active/findings.md +++ b/planning/active/findings.md @@ -90,6 +90,22 @@ independently found B1**, which was the one real bug. | AC1–AC3 invariants | `mad_check.R`: classify only, overlay off, stream vs waterbody split. | | AC4 verify loop | **added** (`test-lnk_config.R`). | +## Phase 4 — live (2026-10-02) + +Full record: `data-raw/logs/params_method_286/README.md`. +- **No change.** ADMS has 5 species and BULK has 7. On each, the per-species + `streams_habitat` digest after connect is identical across: branch; main + fresh + 0.36.2; main + fresh 0.33.0; and (ADMS) main with `mad_m3s` dropped. +- **mad on ADMS.** On streams, CH/CO/SK always sit within their MAD range, and BT/RB have + 0 km. 63.5 km of BT rearing survives in wetland and 1050/1150 edges. + `.frs_rule_to_sql` turns inheritance off for L/W rules under both models, so this is + intended. +- **Correction to my own first reading:** I reported "rear_out = 0 for all species" from + a `grep -A8` that cut off every row after BT. The re-run shows 222 CH and 128 CO + out-of-range rearing segments, all inside waterbodies. +- **mad on BULK** (no discharge): 0 km of stream spawning and rearing for every species, + and no error. + ## Follow-up issue draft (NOT filed — needs body review) **Title:** `lnk_habitat_validate()` scores `mad` watershed groups as if they were `cw` diff --git a/planning/active/progress.md b/planning/active/progress.md index 861dd090..1abe885a 100644 --- a/planning/active/progress.md +++ b/planning/active/progress.md @@ -9,3 +9,4 @@ - Phase 1: fresh pin -> v0.36.2 (floor 0.35.0); `lnk_preflight_fresh(required_formals=)` asserts `frs_habitat_classify(params_method)`; vignette script off `frs_db_conn()` (fresh 0.36.0 precedence flip). Code-check round 1 clean; it flagged stale text (validator/test comments citing v0.33.0, CLAUDE.md `frs_db_conn` line, a skip message), fixed in the same commit - Phase 2: method CSV in the four base bundles (frozen, not csv-synced: B1 from the plan review and code-check round 2, found by both independently), resolver + config_hash fallback, dictionary + tests, discharge in `log_input` primitives, stale `*_mad_*` docs fixed, verify-clean loop over every bundle. Code-check rounds 2 (B1) and 3 (clean) - Phase 3: `mad_m3s` joined onto working streams; `lnk_pipeline_classify(method_csv=)` passes `params_method`; the stream-order rearing bypass is skipped for a `mad` AOI (G3); tests for capture, fallback, override, the `NA` code, the bypass pair, the join args and the persist shape +- Phase 4: live on local fwapg, scratch schemas `zz286_*`, config `default`. ADMS and BULK `streams_habitat` digests identical across branch / main+fresh 0.36.2 / main+fresh 0.33.0 (and ADMS without the `mad_m3s` column). mad on ADMS: invariants hold, BT/RB lose all stream habitat and keep waterbody-rule rearing. mad on BULK (no discharge): 0 km stream habitat for every species, no error. Evidence `data-raw/logs/params_method_286/` diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md index 0627acc0..f9c7e9cf 100644 --- a/planning/active/task_plan.md +++ b/planning/active/task_plan.md @@ -50,11 +50,11 @@ needs adding there. - [x] `lnk_pipeline_connect` stays as it is. `.frs_run_connectivity` is not method-aware (clustering is on gradient), so record that in findings rather than change it ## Phase 4: live verification (local docker fwapg, scratch schema) -- [ ] State the run decisions (config, scratch schema, WSGs, species) before launching -- [ ] **No-change proof.** Run ADMS with the `default` bundle at HEAD and on the branch, reclassifying on one prepared schema. `streams_habitat` must be byte-identical (digest). The all-`cw` table plus the extra `mad_m3s` column must move nothing -- [ ] Repeat the no-change proof on a second, larger WSG (HORS or BULK) -- [ ] **mad takes effect.** Run a thin bundle (`method_csv`) that sets one WSG with discharge coverage to `mad`. It must run, CO/CH/ST habitat must differ from cw, and BT must get no stream habitat from inheriting rules (fresh's documented behaviour). Record km by species in findings -- [ ] A `mad` WSG with no discharge coverage (BULK has none): confirm and record what happens. Every segment fails the mad rule, so expect zero stream habitat. Surface this, and do not add a workaround +- [x] State the run decisions (config, scratch schema, WSGs, species) before launching +- [x] **No-change proof.** Run ADMS with the `default` bundle at HEAD and on the branch, reclassifying on one prepared schema. `streams_habitat` must be byte-identical (digest). The all-`cw` table plus the extra `mad_m3s` column must move nothing +- [x] Repeat the no-change proof on a second, larger WSG (HORS or BULK) +- [x] **mad takes effect.** Run a thin bundle (`method_csv`) that sets one WSG with discharge coverage to `mad`. It must run, CO/CH/ST habitat must differ from cw, and BT must get no stream habitat from inheriting rules (fresh's documented behaviour). Record km by species in findings +- [x] A `mad` WSG with no discharge coverage (BULK has none): confirm and record what happens. Every segment fails the mad rule, so expect zero stream habitat. Surface this, and do not add a workaround ## Phase 5: docs + follow-ups - [ ] RUNBOOK §7 "Where habitat thresholds live": add the method table, the fallback, and that `mad_m3s` exists only in the working schema From 421b40b6876161fee14f196abc22049485b3c1d9 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:27:36 -0700 Subject: [PATCH 6/9] Document the per-WSG cw/mad model in RUNBOOK (#286) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- CLAUDE.md | 3 +- RUNBOOK.md | 53 ++++++++++++++++++++++++++++++++++-- planning/active/progress.md | 1 + planning/active/task_plan.md | 6 ++-- 4 files changed, 56 insertions(+), 7 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 488b78ef..6852c9e7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,7 +76,8 @@ none. The `bcfishpass` copy is a frozen parity input. Runs log the values in `.log_parameters_habitat_thresholds`. `default_tuned` (thin, `extends: default`) is where #284's calibrated CH/BT values land. RUNBOOK §7 "Where habitat thresholds live" has the details, including which columns are -carried but never applied on link's rules path (MAD, edge types) and that +carried but never applied on link's rules path (edge types; MAD only in groups a +bundle's `parameters_habitat_method.csv` puts on `mad`, #286) and that `rear_lake_ha_min` needs a rules rebuild. **`extends:` was broken for provenance until a bundle actually used it.** diff --git a/RUNBOOK.md b/RUNBOOK.md index 67b074a5..eea6019b 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -657,9 +657,9 @@ so a threshold change moved the parity reference too. it (`source` is fresh, not bcfishpass). Tune in `default_tuned`, never here. - **A bundle that declares none falls back to fresh's copy**, with a message, and its `config_hash` then covers that file as `fresh:parameters_habitat_thresholds.csv`. -- **Not every column is live on link's path.** Rules inherit only gradient and - channel width. `*_mad_*` are carried but never applied (MAD is rule-level only, - and streams carry no `mad_m3s`, fresh#114); `spawn/rear_edge_types` are read only +- **Not every column is live on link's path.** Rules inherit gradient and channel + width; `*_mad_*` replace channel width only for watershed groups the bundle's + method table puts on `mad` (next section); `spawn/rear_edge_types` are read only on fresh's no-rules fallback. `rear_lake_ha_min` is **baked into `rules.yaml`** by `lnk_rules_build(thresholds =)`, so changing it needs a rules rebuild. The dictionary's `consumed_by` column has the file:line for each. @@ -673,6 +673,53 @@ so a threshold change moved the parity reference too. child bundle, not the working directory. Before #282 no shipped bundle extended anything, and all three of those were broken. +### Channel width or discharge, per watershed group (#286) + +Each bundle also declares `parameters_habitat_method.csv` (`watershed_group_code`, +`model` = `cw` | `mad`), and `lnk_pipeline_classify()` passes it to +`fresh::frs_habitat_classify(params_method =)`. That needs fresh >= 0.35.0; the +preflight asserts the argument (`required_formals`), not the version, so **a cypher +image still on an older fresh hard-fails at preflight — re-prep hosts before the +next dispatch.** + +- **Every shipped bundle is all `cw`**: a frozen copy of bcfishpass + `parameters/example_newgraph` (188 groups). Outputs did not change (ADMS and BULK + `streams_habitat` digests identical against main on fresh 0.33.0 and 0.36.2, + `data-raw/logs/params_method_286/`). +- **Frozen, not csv-synced.** Its provenance `source` is deliberately not the + bcfishpass URL: `sync_bcfishpass_csvs.R` syncs and auto-merges any entry with + that exact source, which would let an upstream `cw` → `mad` flip change + `default` unreviewed (and miss the two `default_*breaks` bundles). + `derived_from` carries the upstream path. Moving a group is a reviewed row edit, + which moves `config_hash`. +- **Fallback and override** mirror the thresholds: no declaration means fresh's + copy, hashed as `fresh:parameters_habitat_method.csv`; `method_csv =` on the call + wins over both. A group the table does not list is `cw`. +- **`mad_m3s` is joined onto the working streams** (`working_.streams`, from + `whse_basemapping.fwa_stream_networks_discharge` on `linear_feature_id`) for + every group, survives breaking, and is **never persisted** (`cols_streams` does + not carry it). A DB without the discharge table now fails prepare for every + group. `log_input` fingerprints it. +- **What `mad` changes** (fresh's rules, after bcfishpass): stream rules that + inherit thresholds test `mad_m3s` against `*_mad_min/max` instead of channel + width, so species with no MAD thresholds (BT, GR, KO, RB) lose all stream + habitat; the rule-level `channel_width` river-polygon bypass is ignored; SK/KO + lake rearing is polygon membership. **Lake, wetland and `thresholds: false` + rules inherit nothing under either model**, so BT keeps its wetland and + 1050/1150-edge rearing in a `mad` group (ADMS: 63.5 km, all inside + waterbodies). link also skips the stream-order rearing bypass + (`frs_order_child`) for a `mad` group — bcfp applies it in its cw branch only. + fresh does not implement bcfp's `stream_order >= 8` spawning bypass. +- **Discharge coverage is uneven.** A segment with NULL `mad_m3s` fails every mad + test, so a `mad` group without coverage loses all of its stream habitat with no + error. BULK has none in the local fwapg. Check `count(mad_m3s)` before moving a + group. +- **`lnk_habitat_validate()` is still cw-only**: its miss-reason relaxation + rewrites `s.channel_width`, so it scores a `mad` group as if it were `cw`. +- **Connectivity reads no size model.** `.frs_run_connectivity` takes no + `params_method`; its one width test (`.frs_connected_waterbody`, + `spawn_connected_cw_min`) is 0 for SK/KO in every bundle, so it is inert. + --- ## 8b. Drainage closure: never hand-roll it from ltree diff --git a/planning/active/progress.md b/planning/active/progress.md index 1abe885a..34b7cf8f 100644 --- a/planning/active/progress.md +++ b/planning/active/progress.md @@ -10,3 +10,4 @@ - Phase 2: method CSV in the four base bundles (frozen, not csv-synced: B1 from the plan review and code-check round 2, found by both independently), resolver + config_hash fallback, dictionary + tests, discharge in `log_input` primitives, stale `*_mad_*` docs fixed, verify-clean loop over every bundle. Code-check rounds 2 (B1) and 3 (clean) - Phase 3: `mad_m3s` joined onto working streams; `lnk_pipeline_classify(method_csv=)` passes `params_method`; the stream-order rearing bypass is skipped for a `mad` AOI (G3); tests for capture, fallback, override, the `NA` code, the bypass pair, the join args and the persist shape - Phase 4: live on local fwapg, scratch schemas `zz286_*`, config `default`. ADMS and BULK `streams_habitat` digests identical across branch / main+fresh 0.36.2 / main+fresh 0.33.0 (and ADMS without the `mad_m3s` column). mad on ADMS: invariants hold, BT/RB lose all stream habitat and keep waterbody-rule rearing. mad on BULK (no discharge): 0 km stream habitat for every species, no error. Evidence `data-raw/logs/params_method_286/` +- Phase 5: RUNBOOK §7 "Channel width or discharge, per watershed group" (incl. cypher re-prep), CLAUDE.md pointer; follow-up issue body drafted in findings, not filed diff --git a/planning/active/task_plan.md b/planning/active/task_plan.md index f9c7e9cf..651c121b 100644 --- a/planning/active/task_plan.md +++ b/planning/active/task_plan.md @@ -57,9 +57,9 @@ needs adding there. - [x] A `mad` WSG with no discharge coverage (BULK has none): confirm and record what happens. Every segment fails the mad rule, so expect zero stream habitat. Surface this, and do not add a workaround ## Phase 5: docs + follow-ups -- [ ] RUNBOOK §7 "Where habitat thresholds live": add the method table, the fallback, and that `mad_m3s` exists only in the working schema -- [ ] NEWS entry (the version bump comes at merge, through `/gh-pr-merge`) -- [ ] Draft the follow-up issue body in findings, for review and not filed: `lnk_habitat_validate()`'s width relaxation (`R/lnk_habitat_validate.R:759-781`) is channel-width only and gives wrong miss reasons for `mad` groups. `frs_habitat_predicates(model=)` exists to fix it +- [x] RUNBOOK §7 "Where habitat thresholds live": add the method table, the fallback, and that `mad_m3s` exists only in the working schema +- [x] NEWS text drafted in the PR body. This repo writes NEWS.md only in the `Release vX` commit `/gh-pr-merge` makes, so the branch does not touch it +- [x] Draft the follow-up issue body in findings, for review and not filed: `lnk_habitat_validate()`'s width relaxation (`R/lnk_habitat_validate.R:759-781`) is channel-width only and gives wrong miss reasons for `mad` groups. `frs_habitat_predicates(model=)` exists to fix it ## Validation From 7809e3bd2d65b0fbe930bbc3a2ed772ae058d378 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:29:06 -0700 Subject: [PATCH 7/9] Brace multi-line mock bodies in #286 tests Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- tests/testthat/test-lnk_pipeline_classify.R | 7 +++++-- tests/testthat/test-lnk_preflight_fresh.R | 4 +++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/tests/testthat/test-lnk_pipeline_classify.R b/tests/testthat/test-lnk_pipeline_classify.R index 1e2b2bcb..b5bd2ee8 100644 --- a/tests/testthat/test-lnk_pipeline_classify.R +++ b/tests/testthat/test-lnk_pipeline_classify.R @@ -199,9 +199,12 @@ bypass_calls <- function(model) { .lnk_pipeline_classify_build_breaks = function(...) invisible(NULL) ) called <- character(0) + bypass_rule <- list(channel_width_min_bypass = + list(stream_order_parent_min = 5L)) local_mocked_bindings( - frs_params = function(...) list(BT = list(rules = list(rear = list( - list(channel_width_min_bypass = list(stream_order_parent_min = 5L)))))), + frs_params = function(...) { + list(BT = list(rules = list(rear = list(bypass_rule)))) + }, frs_habitat_classify = function(...) invisible(NULL), frs_order_child = function(conn, ..., species) { called <<- c(called, species) diff --git a/tests/testthat/test-lnk_preflight_fresh.R b/tests/testthat/test-lnk_preflight_fresh.R index 0f8731ad..c438cecb 100644 --- a/tests/testthat/test-lnk_preflight_fresh.R +++ b/tests/testthat/test-lnk_preflight_fresh.R @@ -51,7 +51,9 @@ test_that("a pre-0.35.0 frs_habitat_classify signature is caught (#286)", { ns$frs_habitat_classify <- function(conn, table, to, species, params, params_fresh, gate, label_block, barrier_overrides, overwrite, - verbose) NULL + verbose) { + NULL + } expect_identical( .lnk_fresh_missing_formals(ns, .lnk_fresh_required_formals()), "frs_habitat_classify(params_method)") From 39bc0ca478322d5ded7bb6dc9b3fcb318f835192 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:33:34 -0700 Subject: [PATCH 8/9] Archive planning files for issue #286 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- .../2026-10-issue-286-params-method/README.md | 51 +++++++++++++++++++ .../findings.md | 0 .../progress.md | 0 .../review-p1-round1.md | 0 .../review-p23-round2.md | 0 .../review-plan.md | 0 .../review-round3.md | 0 .../task_plan.md | 8 +-- 8 files changed, 55 insertions(+), 4 deletions(-) create mode 100644 planning/archive/2026-10-issue-286-params-method/README.md rename planning/{active => archive/2026-10-issue-286-params-method}/findings.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/progress.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/review-p1-round1.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/review-p23-round2.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/review-plan.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/review-round3.md (100%) rename planning/{active => archive/2026-10-issue-286-params-method}/task_plan.md (90%) diff --git a/planning/archive/2026-10-issue-286-params-method/README.md b/planning/archive/2026-10-issue-286-params-method/README.md new file mode 100644 index 00000000..f1cb53c6 --- /dev/null +++ b/planning/archive/2026-10-issue-286-params-method/README.md @@ -0,0 +1,51 @@ +## Outcome + +Every shipped bundle now declares `parameters_habitat_method.csv`, a per-watershed-group +`cw`/`mad` table. `lnk_pipeline_classify()` hands it to +`fresh::frs_habitat_classify(params_method =)`, so a config can put a group on +bcfishpass's discharge model and have it take effect. +- **The table:** a frozen copy of bcfishpass `example_newgraph`, all `cw`. +- **`mad_m3s`:** joined onto the working streams only; the persist shape is unchanged. +- **Provenance:** `config_hash` carries it, plus a `log_input` fingerprint of the + discharge table. +- **Pin and preflight:** fresh is pinned at v0.36.2 (floor 0.35.0), and the preflight + now asserts the `params_method` argument rather than a version. +- **Stream-order rearing bypass:** a `mad` group skips it, as bcfishpass does. +- **The near-miss:** recording the copy's provenance `source:` as the bcfishpass URL + would have enrolled it in the weekly csv-sync, which auto-merges. An upstream + `cw`→`mad` flip would then have changed `default` with no review. The plan review and + code-check round 2 both found it independently. +- **Behaviour change from the bump:** fresh 0.36.0 reversed `frs_db_conn()`'s env + precedence, which moved one data-raw script off the tunnel on this machine. That script + now uses `lnk_db_conn()`. +- **Follow-up:** `lnk_habitat_validate()` is still cw-only. The issue body is drafted in + `findings.md` and not filed. + +## Measurement + +- **No change.** On ADMS (5 species) and BULK (7 species), the per-species + `streams_habitat` digests after connect are identical across: branch, main + fresh + 0.36.2, and main + fresh 0.33.0. ADMS also matches main with `mad_m3s` dropped. + Neither the pin jump, the extra column, nor the all-`cw` table moves output. +- **ADMS on `mad`**, stream (non-waterbody) km, cw → mad: + + | species | spawn | rear | + |---|---|---| + | BT | 214.5 → 0 | 346.2 → 0 | + | CH | 128.3 → 106.4 | 169.8 → 132.4 | + | CO | 162.1 → 140.1 | 169.8 → 172.3 | + + No CH/CO/SK stream segment falls outside its MAD range. BT keeps 63.5 km of rearing in + waterbody rules, which inherit no thresholds under either model. +- **A wrong turn, kept:** I first reported zero out-of-range rearing from a truncated + `grep`. The full output shows 222 CH and 128 CO segments, all inside waterbodies. +- **BULK on `mad`:** no discharge coverage, so 0 km of stream habitat for every species, + and no error. This is why coverage has to be checked before a group is moved. + +## Evidence + +`data-raw/logs/params_method_286/*` (harnesses + per-run outputs, 2026-10-02 UTC). +Reviews: `review-plan.md`, `review-p1-round1.md`, `review-p23-round2.md`, `review-round3.md`. +Mechanics: RUNBOOK §7 "Channel width or discharge, per watershed group". + +Closed by: PR for #286 (branch `286-thread-fresh-params-method-per-wsg-cw-ma`) diff --git a/planning/active/findings.md b/planning/archive/2026-10-issue-286-params-method/findings.md similarity index 100% rename from planning/active/findings.md rename to planning/archive/2026-10-issue-286-params-method/findings.md diff --git a/planning/active/progress.md b/planning/archive/2026-10-issue-286-params-method/progress.md similarity index 100% rename from planning/active/progress.md rename to planning/archive/2026-10-issue-286-params-method/progress.md diff --git a/planning/active/review-p1-round1.md b/planning/archive/2026-10-issue-286-params-method/review-p1-round1.md similarity index 100% rename from planning/active/review-p1-round1.md rename to planning/archive/2026-10-issue-286-params-method/review-p1-round1.md diff --git a/planning/active/review-p23-round2.md b/planning/archive/2026-10-issue-286-params-method/review-p23-round2.md similarity index 100% rename from planning/active/review-p23-round2.md rename to planning/archive/2026-10-issue-286-params-method/review-p23-round2.md diff --git a/planning/active/review-plan.md b/planning/archive/2026-10-issue-286-params-method/review-plan.md similarity index 100% rename from planning/active/review-plan.md rename to planning/archive/2026-10-issue-286-params-method/review-plan.md diff --git a/planning/active/review-round3.md b/planning/archive/2026-10-issue-286-params-method/review-round3.md similarity index 100% rename from planning/active/review-round3.md rename to planning/archive/2026-10-issue-286-params-method/review-round3.md diff --git a/planning/active/task_plan.md b/planning/archive/2026-10-issue-286-params-method/task_plan.md similarity index 90% rename from planning/active/task_plan.md rename to planning/archive/2026-10-issue-286-params-method/task_plan.md index 651c121b..7872f54a 100644 --- a/planning/active/task_plan.md +++ b/planning/archive/2026-10-issue-286-params-method/task_plan.md @@ -63,7 +63,7 @@ needs adding there. ## Validation -- [ ] Tests pass -- [ ] `/code-check` clean on each commit -- [ ] PWF checkboxes match landed work -- [ ] `/planning-archive` on completion +- [x] Tests pass: full suite FAIL 0 / PASS 2298 / WARN 16 (the warning baseline). `devtools::check()` 0 errors; its 3 warnings and 3 notes are pre-existing, and no added line is non-ASCII. Lintr: the two brace lints on added lines are fixed; the rest follow the file's existing indent style or are stale-install object_usage +- [x] `/code-check`: 3 rounds across the branch, not 3 per commit. Round 1 (Phase 1) was clean, round 2 (Phases 2–3) found B1, and round 3 (cumulative, mechanism-directed) was clean. Plus one Plan review. 4 review agents + 1 plan agent +- [x] PWF checkboxes match landed work +- [x] `/planning-archive` on completion From aa81de3288414fe46bbd4c1d49882d6ee6ebf612 Mon Sep 17 00:00:00 2001 From: almac2022 Date: Thu, 1 Oct 2026 23:33:46 -0700 Subject: [PATCH 9/9] CLAUDE.md status for #286 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx --- CLAUDE.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 6852c9e7..62297ec6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,6 +11,29 @@ Experimental package — breaking all the time and loving the learning curve. St **Prefix:** `lnk_` **Branch:** `main` (current version: `DESCRIPTION` / [`NEWS.md`](NEWS.md)) +## Status (2026-10-01) — per-WSG `cw`/`mad` habitat model threaded (#286) + +**Each bundle's `parameters_habitat_method.csv` reaches fresh as `params_method`.** +- **The table:** all `cw`, a frozen copy of bcfishpass `example_newgraph`. Putting a + group on `mad` is a reviewed row edit. +- **`mad_m3s`:** on the working streams only, never persisted. +- **fresh pin:** v0.36.2 (floor 0.35.0). Cyphers must be re-prepped before the next + dispatch: the preflight asserts the argument and hard-fails otherwise. +- **Mechanics:** RUNBOOK §7 "Channel width or discharge, per watershed group". Evidence: + `data-raw/logs/params_method_286/`. + +**Facts not worth re-deriving:** +- **Never give a bundle file `source: https://github.com/smnorris/bcfishpass` unless you + want it csv-synced.** `sync_bcfishpass_csvs.R` selects on that exact string and + auto-merges byte drift. A frozen copy uses another `source` plus `derived_from`. +- **A `mad` group with no discharge loses all stream habitat silently.** BULK has none. + Waterbody rules (L/W, `thresholds: false`) inherit nothing under either model, so BT + keeps wetland rearing under `mad`. +- **fresh 0.36.0 reversed `frs_db_conn()`'s precedence** (`PG*` first). `lnk_db_conn()` + still reads `PG_*_SHARE` first; on a machine with both groups set they connect to + different databases. +- **`lnk_habitat_validate()` is cw-only**; a follow-up is drafted in the #286 archive. + ## Status (2026-09-29) — #284 step 5: BT `rear_gradient_max` 0.1349 scored and held **Threshold variants are scored on one shared segmentation, never on two full runs.**