Skip to content

Promote recession-analysis fixes and stable nonlinear fitting - #12

Merged
mgcooper merged 31 commits into
mainfrom
dev
Oct 29, 2026
Merged

mgcooper merged 31 commits into
mainfrom
dev

Conversation

@mgcooper

Copy link
Copy Markdown
Owner

Promote the accumulated development fixes to main without a release tag. This includes recession-analysis corrections, event and calendar handling, plotting helpers, reference documentation, refusal of unsupported sloped requests, and descriptive bootstrap percentiles.

The latest repair scales event inputs before nonlinear fitting and rejects nonconverged curve fits. Coefficients and confidence limits return in their original units. The power-law model and event-selection rules are unchanged. The software regression reference is 1.3650 with the existing absolute tolerance of 0.0005; old observations and failed runs remain in history.

Source: dev at 7c27e44ad3228ba9d1532e445ecb774210591a7e.
Base: main at c1bb21bfeb67ebd4f089f5d9a4a47a02bfdae9ae.
Candidate tree: 4375d0b66447535fba2c2b677515bba122434a97.
The cumulative comparison contains 31 commits and 154 changed paths; the latest four-file repair is only part of that range.

Validation:

  • Public Linux product CI: 611/611 passed, including the Kuparuk regression and all ten new conditioning cases.
  • Native R2025b Mac suite: 613/613 passed. The two extra tests are Symbolic-dependent theory demos unavailable in the Linux toolbox set.
  • Eight new conditioning regressions fail on the prior implementation. R2025b Mac/Linux and R2026a Mac Kuparuk captures agree within 8e-11.
  • Configured reviews accepted the predecessor scopes and current repair.

R2024b fit-stage qualification remains separate. Existing demo outputs are unchanged. This promotion does not regenerate or adopt publication results, recover an original paper run, establish calibrated interval coverage, or create a version, tag, release or Zenodo publication.

Main requires one approving PR review. Use the normal merge-commit route only after that rule and the applicable checks are satisfied.

A desktop test run printed a graphics array for every reference-line
label, warned twice about expected conditions, and opened and closed a
window for each test.

drawarrow assigned its output whether or not the caller asked for it,
and none of the five call sites in gpfitb, plplotb and fitphidist ended
with a semicolon, so each label printed its handles.

The fitphi warnings carried no identifier, so nothing could suppress
them. They now carry baseflow:fitphi:incompatibleLateTimeSolution and
baseflow:fitphi:lateTimeSolutionImpliesB, and the tests that ask for the
solution pair fitphi falls back from, or that run a fit to the iteration
limit of nlinfit, suppress the warning they expect. A user still sees it.

runtests draws test figures off screen for the length of the run and
puts the figure default back on both exits. A figure created with an
explicit 'Visible' of 'on' ignores that default, so fitphidist takes its
visibility from the new private figurevisibility, which answers off
whenever the root asks for invisible figures.

The version banner carried the digits of 1.1.0. It now takes them from
the version string, so the art cannot fall behind again.

test_pointcloudplot asserted a figure position it had asked for rather
than the one the window manager granted, so the supplied-axes test
failed on a display. It reads the position back, the sibling position
tests assert only the size, and the new test_sizefigure covers the kept
position on the private helper itself.
Three tests made a panel current with axes(), which shows and raises its
figure whatever the root default asks for, so a window still opened
during a run. They name the current axes through the figure instead. A
root listener over the whole suite now reports no visible figure.

The demos printed bare results, which reach the screen as ans. Each
value now prints with its name. baseflow_demo_3 reports k in cm s-1,
the unit aquiferprops returns it in.

The large drawn letters of the version banner read 1.0.0 and now read
1.2.0. That art is drawn by hand, unlike the two slots beside and below
the v, which come from the version string; a comment says which is
which, and test_internal reads the banner back and checks both slots
against the version.

The generated demo pages keep the output they were built with. They are
rebuilt the next time the docs are rebuilt for their own reason.
Move each function-level test of tests/TestBaseflow.m to the suite of
its function, tests/test_<function_name>.m. New suites: test_getstring,
test_basinname, test_setopts, test_fitab, test_aquiferstorage,
test_aquiferthickness, test_eventfinder, and test_islocalmin.
test_islocalmax becomes a parameterized classdef that holds its peak
test and the islocalmax rows. test_getdqdt, test_fitevents,
test_plotdqdt, test_conversions, and test_plfitb take the other rows.

A method named like its class is the class constructor, which makes the
suite builder skip the file, so the moved methods take descriptive
names. TestBaseflow keeps the event-detection checks of getevents,
eventfinder, and wrapevents. Each earlier row maps to one row.
The mean tau of a Pareto tail, tau0*(2-b)/(3-2*b), diverges at and
below the tau pole at alpha = 2. There the formula gave a negative
value: gpfitb reported tauExp = -10.01 for alpha = 1.5. The private
expectedtau returns Inf there and NaN where no Pareto tail exists. It
alone raises baseflow:expectedtau:tauPole, with the count of values of
b that give no finite mean tau. gpfitb, plfitb, plplotb, and
getfunction('expectedTau') call it. The gpfitb and plplotb labels omit
a mean that is not finite.

expectedQ returns NaN for Qexp, Q0, pQexp, and pQ0 when tau is not
finite, rather than 0 or complex values. fdcurve returns one reference
value per point, NaN for a NaN point, and leaves missing flows out of
the plotting positions. A caller that caught
baseflow:plfitb:tauPoleReplicates must catch
baseflow:expectedtau:tauPole.
- specialfunctions raises the Beta function to the power n+3 in the
  late-time Rupp and Selker (2005) constant. So fR2(0) = 4.805, the
  Brutsaert and Nieber (1977) constant. phi from the RS05 solutions
  changes for every b other than 1 and 3/2.
- plfitb puts the xmin sample in the tail mask, as plfit fits x >= xmin.
- fitab 'nls' reports its interval at the confidence level alpha, and
  the Octave path uses the residual degrees of freedom.
- prepfits drops masked and zero-weight points, so every fitab method
  fits the same points.
- The CTS stencils pair each derivative with the flow at the same time;
  ETS leaves a rejected window empty; VTS widens across flat steps.
- pointcloudintercept orders its a bounds and pivots 'mean' through the
  geometric means.
- dndtuncertainty correlates complete rows; aquiferstorage uses one path
  for scalar and array input; Qnonlin covers b = 1 and an emptied
  reservoir; printtrend scales by the years spanned; conversions takes
  'Nstar'; getfunction handles apply elementwise; gpfitb tau0reps are
  sigma/k; quantreg tests significance on its interval; the fdcurve
  label and the plfitb limit help match the code.
TestBaseflow runs baseflow_demo_kuparuk once and checks each stage
against the stages before it. It checks the recession exponent, the
globalfit tail, mean tau, and expected flows, and the baseflow trend at
the pQexp quantile. It also checks the thickness from that trend, the
uncertainty coverage factor, and the BS04 and RS05 aquifer properties
at b = 3/2. A synthetic set of exact recessions checks the
event fits and the tau sample against the known recession, and the
divergent case: Inf mean tau and NaN expected flows.

baseflowtrend requires pctl in (0,1). A NaN pctl, which a divergent
mean tau gives through pQexp, made trendplot fit the mean flow.

tests/test_corechain.m is removed. Its workflow checks are in
TestBaseflow. Its function checks are in test_gpfitb, test_plplotb,
test_fitphi, test_fitphidist, test_dndtuncertainty, and test_cloudphi.
Its aQbString check duplicates test_eqstrings.
- setopts and globalfit default drainagedensity to NaN. The default of
  0.8 overrode a streamlength given alone with A*0.8/1000 and warned.
  globalfit compares the two only when both are given, with a sqrt(eps)
  relative tolerance, so A*Dd/1000 computed in another order raises no
  warning. The warning has the identifier
  baseflow:globalfit:inconsistentStreamlength. L does not enter the
  flat phi solutions, so no result changes.
- dndtuncertainty takes the years that T spans directly. It reshaped
  the padded record into 365-day rows, which a leap day breaks. The
  result on a no-leap calendar is unchanged.
- TestBaseflow checks both on the Kuparuk chain: a streamlength given
  alone or within rounding raises no warning and an inconsistent pair
  warns; a calendar with leap days gives the no-leap uncertainty.
- cloudphi uses userab = [a b] as the late-time line and fits the
  early-time line to the point cloud. Its userab branch was empty, so
  a call with userab failed.
- fitphidist leaves a phi outside [0, 1] out of the beta fit and warns
  baseflow:fitphidist:phiOutsideUnitInterval with the count. betafit
  errored on such a value, so globalfit 'distfit' failed. 'phicombo'
  shares this check.
- fitab 'mean' with no order fits the slope-1 line it documents and
  returns b = 1; it returned NaN. 'median' and 'envelope' document no
  default slope, so they raise baseflow:fitab:orderRequired.
- aquiferstorage rounds b with round(100*b)/100, which Octave supports.
  tests/octave_smoke.m checks the storage at b = 3/2.
getfR1 divides the root mu by 4-2*gamma, as equation 33 of Rupp and
Selker (2005) prints. It divided by 4*(1-2*gamma). That gave 0.869 at
n = 0 for the published 1.108, a pole at n = 1, and negative values
above n = 2.4. fR1/(n+1) is their Phi1 (eqs 54 and 56), and fR2/2 is
their Phi2 (eq 59). phi from the RS05 solutions changes again for every
b other than 1 and 3/2.

test_specialfunctions sweeps the seven rows of their Table 1 for both
constants, within half a unit of the printed fourth significant figure.
'mean', 'median', and 'envelope' fix the slope of the line, and each
uses slope 1, the linear reservoir, when order is not given. Without
order, 'median' raised a signrank error or baseflow:fitab:orderRequired,
and 'envelope' returned NaN or the same error.

test_fitab checks each method against the definition: log a is the
center of log -dq/dt minus the center of log q, with the mean or the
median as the center.
aquiferstorage rounds b to two decimals to pick the linear, exponential,
or power-law storage, and the power-law formulas use the unrounded b.
At b = 1.567 the rounded b changed the storage by +2.1% at q = 0.01 and
-3.4% at q = 1e6. test_aquiferstorage checks the closed form at
b = 1.567.
makedocs built the search database before it published the doc pages
and the m2html function pages, so the database indexed stale pages.
makedocs builds the database last.
Output of baseflow.internal.makedocs('docpages', 'functions',
'docsearch'). The pages follow the current help text, including the
drainagedensity default of NaN. The demo pages are not rebuilt.
custom_toolbox.json keeps its earlier content, because the build writes
the absolute path of the build folder into it.
- tests/TestBaseflow.m: the check that globalfit recovers the Kuparuk
  recession exponent is test_kuparukRecessionExponent. Its expected
  value is the property kuparukb. The help and comments name the annual
  workflow and the 1.1.0 release notes as the source of the value. The
  expected value and the tolerance do not change.
- tests/TestBaseflow.m: the class setup runs the chain of
  baseflow_demo_kuparuk with the demo options, and its comment says so.
  The aquifer-property check follows the steps of baseflow_demo_3 on the
  point cloud of that chain, and its comment says so. The setup names
  its figure snapshot setupfigs, so it does not shadow the figsbefore
  property.
- CHANGELOG.md: the [Unreleased] TestBaseflow entry describes the same
  checks and names the suites that hold the checks of the removed
  test_corechain. The expectedtau entry says that no function raises
  baseflow:plfitb:tauPoleReplicates.
- TODO.md: an update names the test that holds the zero-figure check of
  the removed test_corechain.
- toolbox/docs/html/custom_toolbox.json: Git does not track it.
  builddocsearchdb writes it again on each docs build, with the absolute
  path of the build folder, so a copy names a folder of the machine that
  built it. The help registration reads toolbox/info.xml. A local search
  server returned the same query results with the file tracked,
  missing, unreadable, or naming another folder.
- .gitignore and toolbox/toolbox.ignore: rules keep the file out of
  version control and out of a toolbox package. The search database in
  helpsearch-v4_en still ships.
- tests/test_makedocs.m: checks that a toolbox package leaves out the
  file and holds every file of the search database, and that Git
  ignores the file.
- CHANGELOG.md: an [Unreleased] entry describes the change.
* feat/recession-analysis-fixes:
  fix(docs): stop tracking the generated doc search metadata
  test(baseflow): describe the Kuparuk workflow checks as they run
  docs: regenerate the doc pages, function pages, and search database
  fix(internal): build the doc search database after the pages
  fix(baseflow): round b in aquiferstorage only to choose the formula
  fix(baseflow): fit slope 1 by default for fitab 'median' and 'envelope'
  fix(baseflow): correct the RS05 early-time constant fR1
  fix(baseflow): repair cloudphi userab, phi domain, fitab order, rounding
  fix(baseflow): keep a given streamlength and accept leap-day calendars
  test(baseflow): make TestBaseflow an end-to-end workflow suite
  fix(baseflow): correct the RS05 late-time constant, tail mask, and fits
  fix(baseflow): return Inf for a divergent mean tau and NaN expected flow
  test(baseflow): split TestBaseflow into per-function suites
  docs: refresh workflow and local style guidance

Outcome: dev gains the repairs of the RS05 constants fR1 and fR2 and
of the plfitb tail mask that globalfit uses. Further repairs cover the
divergent mean tau and expected flow, cloudphi, fitab, and
aquiferstorage. It also gains a given streamlength in globalfit, leap-day calendars in
dndtuncertainty, and the TestBaseflow workflow suite with
per-function suites. The docs build indexes the pages it publishes,
the doc and function pages are regenerated, and Git does not track
the generated doc search metadata.

Compatibility: globalfit phi changes for every b other than 1 and 3/2.
fitab 'mean', 'median', and 'envelope' fit slope 1 when order is not
given. The mean-tau pole warning has the identifier
baseflow:expectedtau:tauPole. A docs build writes
toolbox/docs/html/custom_toolbox.json, which Git ignores. The demo
HTML pages and live scripts are unchanged; this merge rebuilds no
demo output.

Verification: the merge tree equals the tree of the source tip
38d9ad3 (becbd22). The full suite passed 539 of 539, with 0 failed,
0 incomplete, and 0 open figures. That run came before two review
repairs that changed only tests/test_makedocs.m and CHANGELOG.md.
test_makedocs passed again on the source tip. It fails when
toolbox.ignore or its rule is lost, or when the file is tracked again.

Reviewed range: 5964528..38d9ad3, 14 commits. No ExecPlan or final
report governs this work.
- toolbox/+baseflow/private/plotrain.m: one helper draws the rain
  circles for pointcloudplot and plotdqdt. Each circle has the size of
  the plotted marker plus sqrt(pi*rain^2). The helper keeps the
  comments of both local copies, which it replaces.
- plotrain returns nan when no point has rain > 0. NaN rain gives a
  NaN sum, so the sum test alone drew no circle and left the output
  unassigned. For dry or NaN rain, the plotdqdt struct has hrain = nan
  where it had no hrain field. The legend and the figure do not change.
- tests/test_plotrain.m: both callers draw the same circles at the wet
  points, and zero or NaN rain draws none. The circle rows also pass
  on the code before this change.
- toolbox/+baseflow/hyetograph.m: the help names the order of H and the
  tags HyetographFigure, HyetographAxis, StreamflowPlot, and
  PrecipitationPlot. The examples search from the figure H(1), because
  a search of all of H returns the streamflow axes twice. The untagged
  precipitation axes is ancestor(H(4), 'axes').
- tests/test_hyetograph.m: each documented lookup returns its one
  element of H, and the ancestor of the bars is the reversed axes.
- CHANGELOG.md: an [Unreleased] Fixed entry describes the help.
Index each month and day assignment before removing February 29. Cover complete leap, non-leap and two-year calendars with datetime and datenum inputs, preserving timestamp, flow and rainfall alignment. Closes bfra-kwr.27.
Retain clone and Setup installation. Document the explicit repository URL and numeric-tag MPI route only after a release includes pathList.m and passes a clean-checkout install check. Add the plain toolbox path list; no existing release is install-qualified. BFRA consumer of matfunclib-juq.47, tracked by bfra-7gr.
Append the missing direct Project UUID exclusion while preserving the existing ignore prefix and all tracked or nested Project membership files. No untracking or regeneration. BFRA consumer of matfunclib-juq.49, tracked by bfra-7gr.
Rebuild only functions and docsearch from the reviewed source. Update hyetograph handle examples, the shared plotrain callers and wrapevents calendar help. Preserve all demo HTML, live scripts and numerical products. Closes bfra-kwr.26.1; release and demo-regeneration choices remain separate.
- toolbox/+baseflow/+deps/peakfinder.m: with the endpoints excluded, the
  end check of a BFRA-local branch could add the last candidate peak
  when no candidate had passed the selectivity, and then read an unset
  tempLoc. The branch starts with no candidate and adds one only when
  one exists. Calls that did not error give the same result, because
  the branch reads tempLoc only when a candidate exists.
- tests/test_peakfinder.m: a selectivity of 1 on a series that ends
  below its extrema finds no peak; the code before this change errors
  on that input.
- CHANGELOG.md: an [Unreleased] Fixed entry describes the fix.
eventfinder tests slope and curvature signs, extrema, and trends. On a
straight or constant stretch these are zero in exact arithmetic, but the
computed values carry rounding, and the smoothing in getevents makes it
depend on the MATLAB release, so the selected events did too.

- toolbox/+baseflow/eventfinder.m: each test treats a value as zero
  within the rounding bound of its own computation. With
  E1 = qerror + 2*u*max(abs(q)), a slope is zero within 2*E1 and a
  curvature within 4*E1. The 3-day mean adds 3*u*max(abs(q)). Extrema
  and the endpoint steps use the same thresholds. No-change runs are
  set to exactly 0. A new option, 'qerror', takes the rounding bound of
  the input values; without it, eventfinder counts one rounding.
- private smoothflow.m returns the bound of its smoothing,
  2.25*(win+1)*u*max(abs(x)), and getevents and findevents pass it.
- private islocalmax.m and islocalmin.m take a peakfinder selectivity.
  islineconvex.m and islinepositive.m take a rounding bound, and calls
  without one keep the backslash slope. The new private trendsign.m
  holds the bounded trend sign.
- toolbox/functionSignatures.json: the qerror option.
- tests/test_eventfinder.m: constant, straight, curved, and
  near-threshold series at three flow scales. The earlier rule fails
  every rounding-level case.
- tests/TestBaseflow.m: kuparukb is 1.3465 with btol 5e-4. R2025b gives
  1.346467 and R2026a 1.346470. R2024b selects the same 231 events but
  gives 1.3577, because its per-event fits differ.
- CHANGELOG.md: an [Unreleased] Fixed entry describes the change.
Reject unsupported sloped-porosity requests before scientific calculations.
Keep the supported flat-aquifer path unchanged. Add descriptive percentile
summaries from retained bootstrap replicates without changing Fits or RNG.

The two source commits passed 86 affected MATLAB R2025b cases, nine separate
Octave assertions and changed-file lint. Configured implementation and
closeout reviews accepted the exact source tree. This merge preserves existing WIP.
Original paper-run recovery and scientific uncertainty choices remain open.

Reviewed implementation: 5c298df..bda74bc.
No committed closeout-only suffix; local records retained in stealth mode.
Reports:
  .agents/plans/final-reports/2026-10-24-bfra-kwr4-flat-contract.md
  .agents/plans/final-reports/2026-10-24-bfra-kwr61-bootstrap-percentiles.md

* fix/bfra-contract-percentiles-20261024:
  feat(baseflow): add descriptive bootstrap percentile summaries
  fix(baseflow): reject unsupported sloped porosity requests
Scale event inputs before nonlinear fitting and restore coefficients and
confidence bounds to their original units. Reject nonconverged curve fits
while preserving the solver order and power-law model.

Add ten public-entry conditioning regressions and adopt the owner-approved
Kuparuk software reference 1.3650 with the existing 0.0005 tolerance.

Validation: 613/613 native R2025b tests; R2025b Mac/Linux and R2026a Mac
Kuparuk captures agree. Configured review accepted the four-file candidate.
* fix/bfra-kuparuk-conditioning-20261028:
  fix: stabilize event-scale nonlinear power-law fits

Integrate the accepted event-scale nonlinear-fit conditioning repair.
The reviewed source range is 9c0eaf8..ba668ee.
The staged tree exactly matches the reviewed product tree; no conflict or
merge-only delta exists. Reuse 613/613 native tests and configured review02.
The approved Kuparuk reference is 1.3650 with unchanged 0.0005 tolerance.
Public dev CI remains due after the separately authorized push.

Local report: .agents/plans/final-reports/2026-10-28-bfra-kwr25-public-ci-repair.md
Records are retained in local stealth mode; both primary lint configs preserved.
@mgcooper
mgcooper merged commit feee9c3 into main Oct 29, 2026
5 checks passed

This branch was successfully deployed

1 active deployment
github-pages — 7c27e44a Deployed Oct 28, 2026 by mgcooper via deploy #32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant