diff --git a/.gitignore b/.gitignore index 7a83bcb..eac0085 100644 --- a/.gitignore +++ b/.gitignore @@ -72,6 +72,9 @@ codegen/ *.slxc .MATLABDriveTag +# Written by the Help browser search-database build beside the docs; not a source file. +custom_toolbox.json + # ----------------- # ---- PYTHON ----- # ----------------- @@ -621,3 +624,5 @@ TSWLatexianTemp* # project's state is machine-local, not source. **/resources/* !**/resources/project/ + +**/resources/project/uuid-*.xml diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b8396b..3274f78 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,201 @@ This file lists notable changes to the baseflow toolbox. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); the project uses semantic versioning. +## [Unreleased] + +### Changed + +- The unit tests of `tests/TestBaseflow.m` live in per-function suites, + `tests/test_.m`. `TestBaseflow` is the integration suite + of the documented workflow. It runs the chain of `baseflow_demo_kuparuk` + once, with the demo options but without figures or the bootstrap. It + checks each stage against the stage before it. It also checks the + Kuparuk recession exponent, the BS04 and RS05 aquifer + properties at b = 3/2, and a synthetic divergent tail. `TestBaseflow` + holds the workflow checks of the removed `tests/test_corechain.m`, and + the per-function suites hold its function checks. +- One private function, `expectedtau`, computes the mean tau for + `gpfitb`, `plfitb`, `plplotb`, and `baseflow.getfunction`. It alone + raises the pole warning, `baseflow:expectedtau:tauPole`, with the count + of values of b that give no finite mean tau. No function raises + `baseflow:plfitb:tauPoleReplicates`. Suppress + `baseflow:expectedtau:tauPole` to silence the pole warning. +- Git does not track `toolbox/docs/html/custom_toolbox.json`. + `builddocsearchdb`, which `baseflow.internal.makedocs` calls for the + 'docsearch' option, writes it again on each build, with the absolute + path of the build folder. A copy names a folder that exists only on the + machine that built it. The search database in `helpsearch-v4_en` still + ships, and `toolbox/toolbox.ignore` leaves the file out of a toolbox + package. + +### Fixed + +- The `eventfinder` filters treat a slope or curvature as zero when its + magnitude does not exceed the rounding error of its own computation. + Exact arithmetic gives a zero curvature on a straight stretch of flow, + but the computed values carry rounding, and the smoothing that + `getevents` applies makes it depend on the MATLAB release. So the + filters selected different events on different releases. The bound + scales with the flow and, after smoothing, with the smoothing window; + the `eventfinder` help states it. `getevents` passes the bound of its + smoothing through the new `eventfinder` option `'qerror'`, and + unsmoothed input counts one rounding per value. The Kuparuk annual + workflow selects the same 231 events on R2024b, R2025b, and R2026a, + where it selected 229 or 230. Against the earlier R2025b result, the + `rmnochange` filter excludes 46 more late-season days on straight + stretches, which shortens or splits some events. The workflow's + recession exponent b is 1.3465 on R2025b and R2026a, against 1.3541 + in the 1.1.0 release notes, and `TestBaseflow` checks that value. + R2024b selects the same events but gives 1.3577, because its + per-event nonlinear fits differ. +- `baseflow.deps.peakfinder` returns no peak when no extremum passes the + selectivity with the endpoints excluded. It read an unset candidate and + errored for a positive selectivity on a series that ends below its + extrema. +- `wrapevents` removes February 29 before annual grouping and keeps the + remaining timestamps, flow and rainfall rows aligned. Leap-year input + previously failed the complete-year check. +- The `hyetograph` help names the tags the code sets. The figure has the + Name 'Hyetograph' and the Tag 'HyetographFigure'. The streamflow axes is + 'HyetographAxis', and the series are 'StreamflowPlot' and + 'PrecipitationPlot'. The help searched for 'StreamflowAxis' and + 'PrecipitationAxis', which no object has. A search of all of H returns + the streamflow axes twice, so the examples search from the figure H(1). + The precipitation axes has no tag; `ancestor(H(4), 'axes')` finds it. +- The late-time Rupp and Selker (2005) constant in `specialfunctions` + raises the Beta function to the power n+3. At n = 0 it gives 4.805, the + Brutsaert and Nieber (1977) constant 4.804; it gave 1.857. `globalfit` + uses the RS05 solutions by default, so its phi changes for every b + other than 1 and 3/2. `aquiferprops` 'RS05' gives the 'BS04' k at + b = 3/2. fR2/2 matches the Phi2 column of Rupp and Selker (2005) + Table 1 for n from 0 to 64. +- The early-time Rupp and Selker (2005) constant fR1 in + `specialfunctions` divides the root mu by 4-2*gamma, as their equation + 33 prints. It divided by 4*(1-2*gamma), which gave 0.869 at n = 0, a + pole at n = 1, and negative values above n = 2.4. + fR1/(n+1) matches the Phi1 column (m = 1) of their Table 1 for n from 0 + to 64, with 1.108 at n = 0. So `globalfit` phi changes again for every + b other than 1 and 3/2. A result that divides by phi, such as the + thickness from `aquiferthickness` and `aquifertrend`, changes with it. + A storage trend does not depend on phi. +- `plfitb` puts the xmin sample in the tail mask, as `plfit` fits the + tail x >= xmin. So the `globalfit` a and expected flows change. +- `baseflowtrend` requires `pctl` in (0,1). A NaN `pctl`, which a + divergent mean tau gives through `pQexp`, made `trendplot` fit the mean + flow, so the baseflow equalled the mean flow. +- `gpfitb` and `plfitb` return Inf for the mean tau at or below the tau + pole at alpha 2. The mean of a Pareto tail diverges there. The formula + tau0*(2-b)/(3-2*b) is negative below the pole, so `gpfitb` reported a + tauExp of -10.01 for a sample with alpha 1.5. The mean is NaN where no + Pareto tail exists, at alpha 1 or below. +- `gpfitb` and `plplotb` draw no mean-tau label where the mean or its + bounds diverge, so no label names a negative or infinite tau. +- A `plfitb` bootstrap replicate at or below alpha 2 has an infinite tau, + so `tau_sig`, `tau_L`, and `tau_H` are NaN. +- `expectedQ` returns NaN for `Qexp`, `Q0`, `pQexp`, and `pQ0` when tau + is not finite. So `globalfit` reports no expected flow on data with b + at or above 3/2. It returned 0 at b = 3/2 and complex values between + 3/2 and 2, where the negative tau entered a fractional power. +- `fdcurve` returns a reference value for each reference point, and NaN + for a point that is NaN or above every flow. It sized its output from + the first point only. Missing flows stay out of the plotting + positions. +- `baseflow.getfunction('expectedTime')` shares the tau pole. The + `'boftautau0'` and `'aofa0'` handles apply elementwise, so array input + gives one value per element instead of a matrix division. +- `fitab` 'nls' reports its interval at the confidence level `alpha`. + `nlparci` and `regress` take a significance level, so the default 0.68 + gave a 32% interval, and the linear fallback a 95% one. The Octave path + used one degree of freedom. +- `prepfits` drops masked and zero-weight points, so every `fitab` method + fits the same points. 'nls' honors the mask, and the OLS interval and + `Fit.N` count the fitted points only. +- The CTS stencils B2, F2, C2, and C4 pair each derivative with the flow + at the same time. B2 and F2 paired it with a flow half a step away. + That biased a by about 5% at a step of a tenth of tau. +- The ETS method leaves a rejected window empty. It interpolated over + the rejected windows, so a rise inside a recession took the slope of + the windows around it. The example data has no such window, so its + results are unchanged. +- The VTS method widens its window across a flat step, so a record in + whole units gives no zero slopes (Rupp and Selker, 2006). +- `pointcloudintercept` orders its a bounds for flows below 1. Its 'mean' + method pivots through the geometric means of q and -dq/dt, so its + bounds are in flow units. +- `dndtuncertainty` correlates the event variables over the events where + all three exist, so a nan phi from `phifitensemble` leaves the + uncertainty finite. +- `aquiferstorage` uses one path for scalar and array input. The scalar + path with a `Ktime` input returned negative storage for 1 < b < 2. +- `Qnonlin` gives the exponential recession at b = 1, and for b < 1 a + flow that stays 0 once the reservoir empties. `generateTestData` uses + it for every b. +- `printtrend` scales the trend by the years the record spans to report + the total change. It used the sample count. +- `conversions` accepts 'Nstar' as an input, and its help gives k and + alpha from the pdf of tau. +- `gpfitb` returns the bootstrap replicates of tau0 = sigma/k in + `tau0reps`. It returned sigma. +- `quantreg` flags a significant slope when its bootstrap interval + excludes zero. A z test on the replicates divided their spread by the + square root of the replicate count. +- The `fdcurve` x label names the non-exceedance probability it plots, + and the `plfitb` help says that `limit` bounds xmin. +- `gpfitb`, `plplotb` and `fitphidist` print nothing when they label a + figure. The private `drawarrow` assigned its output whether or not the + caller asked for it, and no call site ended with a semicolon, so each + label printed a 1x2 graphics array. +- `baseflow.internal.version` draws the version it reports. The two + slots beside and below the v carried the digits of 1.1.0 and now come + from the version string. The large letters, which are drawn art, read + 1.2.0. +- `baseflow.internal.runtests` draws test figures off screen, so a run on + the desktop opens no window for a test that closes its own figure. It + puts the figure default back on both exits. +- Three tests made an axes current with `axes()`, which shows and raises + its figure whatever the root default asks for, so a window still opened + during a run. They name the current axes through the figure instead. +- The demos print each value with its name. `baseflow_demo_1`, + `baseflow_demo_2` and `baseflow_demo_3` displayed bare results, which + reach the screen as `ans`. +- `fitphidist` draws its figure off screen when the root asks for + invisible figures, through the new private `figurevisibility`. A figure + created with an explicit `'Visible'` of `'on'` ignores that default, so + `showfit` true opened a window during a test run. +- The `fitphi` warnings carry identifiers, + `baseflow:fitphi:incompatibleLateTimeSolution` and + `baseflow:fitphi:lateTimeSolutionImpliesB`, so a caller can suppress or + catch one by name. The tests that ask for a solution pair `fitphi` + falls back from now suppress the warning they expect, as do the tests + that run a fit to the iteration limit of `nlinfit`. +- `cloudphi` uses the `userab` line as the late-time line and still fits + the early-time line to the point cloud. A call with `userab` failed + with an undefined variable. +- `fitphidist` leaves a phi outside [0, 1] out of the beta fit and warns + `baseflow:fitphidist:phiOutsideUnitInterval` with the count. `betafit` + errored on such a value, so `globalfit` 'distfit' failed when one event + gave a phi above 1. 'phicombo' set those values to NaN without a + warning, and it shares the `fitphidist` check. +- `globalfit` keeps a `streamlength` given without `drainagedensity`. + `setopts` and `globalfit` gave `drainagedensity` a default of 0.8, + which replaced that `streamlength` with A*0.8/1000 and warned. The + default is NaN. A `streamlength` within rounding of A*Dd/1000 raises no + warning, and the warning has the identifier + `baseflow:globalfit:inconsistentStreamlength`. The flat-aquifer phi + solutions do not use L, so no result changes. +- `aquiferstorage` runs on Octave. It rounded b with `round(b,2)`, and + the Octave `round` takes one argument. It rounds b to two decimals + only to choose the formula, and the power-law formulas use the + unrounded b. At b = 1.567 the rounded b changed the storage by +2.1% at + q = 0.01 and -3.4% at q = 1e6. +- `dndtuncertainty` accepts a daily calendar with leap days. It reshaped + the record into 365-day rows, which a leap day breaks. It takes the + years that T spans directly, so the result on a no-leap calendar is + unchanged. +- `fitab` 'mean', 'median', and 'envelope' fit a line of slope 1 when + `order` is not given, and return b = 1. 'mean' returned b = NaN, + 'median' failed in `signrank`, and 'envelope' returned NaN. + ## [1.2.0] - 2026-09-17 ### Added diff --git a/README.md b/README.md index d52dfbd..2ac714c 100644 --- a/README.md +++ b/README.md @@ -56,8 +56,7 @@ and `tau` conventions. ## Install -- Clone this repo or use [`mpm`](http://mobeets.github.io/mpm/) in your Matlab terminal: - - `mpm install baseflow` +- Clone this repository. - cd into the `toolbox` directory. - Run `Setup.m`: - Type `msg = Setup('install')` at the command window then press enter. @@ -68,6 +67,19 @@ and `tau` conventions. - To debug, try `runtests('tests', 'Debug', true)`. - In new Matlab sessions, cd to the toolbox directory and try `Setup('addpath')` or just `Setup` to add the toolbox to your search path (or manage the search path however you normally do). +The optional [Matlab Package Installer (MPI)](https://github.com/mobeets/mpm) +route uses this repository's explicit URL and a numeric release tag: + +```matlab +mpi install -u https://github.com/mgcooper/baseflow.git -t +``` + +The selected release must include a root `pathList.m` containing `toolbox`. +It must also pass an installation check from a clean checkout. +No existing release has been qualified for this command. +Use the clone route until a qualified release is published. +Do not install this project by the bare name `baseflow`. + For more options, see [Configuration](#configuration). For Octave compatibility, see [Octave](#octave). @@ -234,4 +246,4 @@ cd baseflow Baseflow recession analysis is a set of methods in hydrologic science used to estimate aquifer properties from streamflow measurements. Properties that cannot be observed, such as hydraulic conductivity, are inferred from something that can be observed, such as streamflow. The underlying theory is based on solutions to the one-dimensional (lateral) groundwater flow equation ("Boussinesq equation"). The theory makes various strong assumptions, including that the aquifer extent is much larger in the lateral direction than the vertical, and that capillarity can be ignored or parameterized by the concept of 'drainable porosity'. This reduces the three-dimensional Richard's equation to one dimension and eliminates unsaturated flow considerations. The theory has been validated with laboratory experiments and may work better than you expect. When applied to entire catchments ("watersheds"), the theory becomes an effective one, meaning the inferred properties are those that give the same solution to the Boussinesq equation that one would obtain for a single hillslope with those same properties and geometric similarity. ---> \ No newline at end of file +--> diff --git a/STYLE.local.md b/STYLE.local.md index 800f490..0e164d9 100644 --- a/STYLE.local.md +++ b/STYLE.local.md @@ -12,13 +12,14 @@ merged into it). - Give a class suite the same name as its file (`test_fitab`). - `TestBaseflow.m` is the integration suite for the toolbox workflow, so it keeps its PascalCase name. +- A suite that tests several functions together keeps a descriptive + name: `test_eqstrings` and `test_internal`. - Variables use short lowercase or camelCase names (`q`, `dqdt`, `gotopath`). ## Formatting -- Indent with 3 spaces; no tabs. -- Wrap at roughly 80 columns; continue long lines with `...`. -- Close every function with an explicit `end`. +Use the shared MATLAB formatting rules in `STYLE.md`. + - Function help uses an uppercase H1 (`%FITAB fit event-scale ...`) followed by `Syntax`, `Description`, input, and output sections for public functions. See `toolbox/+baseflow/fitab.m` for the reference format. @@ -78,3 +79,7 @@ as this: > This method fails when nmin is 0, and possibly when nmin is 1. runlength > returns 1 for consecutive nan values. See isminlength. + +## Other project conventions + +The New-test convention section defines this repository's additional test requirements. diff --git a/STYLE.md b/STYLE.md index 52506d2..87bf33e 100644 --- a/STYLE.md +++ b/STYLE.md @@ -24,17 +24,17 @@ live in `STYLE.local.md`. Match surrounding code first; these are defaults. ## Comments and documentation -Comment discipline is **strict and non-negotiable** — a hard requirement, not a -preference. +Comment discipline is **strict and non-negotiable**. It is a hard requirement, +not a preference. - **Never write uncommented code.** Every new function gets a docstring/header, and every block of code gets an explanatory comment that says *why*, not just *what*. Do not skip a comment because the code looks trivial, simple, obvious, or - self-explanatory — that is not an exception. Uncommented code is incomplete and + self-explanatory. That is not an exception. Uncommented code is incomplete and must not be submitted. - **Never remove existing comments.** A comment may be removed *only* when the code it documents is itself removed. It is never acceptable to delete, trim, shorten, - summarize away, or "clean up" comments — not for brevity, not for tidiness, and not + summarize away, or "clean up" comments, whether for brevity, for tidiness, or because a comment seems obvious or redundant. - **Edit existing comments only for accuracy.** When you change code, update the affected comments and docstrings so they stay correct, preserving the information @@ -88,6 +88,21 @@ project text: - A comment longer than the code it explains. - A sentence that repeats another sentence. +### Check before you finish + +Before you finish a change, check each new or changed comment, docstring, +and prose sentence against these rules: + +- 20 words or fewer for an instruction, 25 or fewer for a description; +- one instruction per sentence; +- the active voice with a named actor; +- verbs, not noun forms of verbs; +- noun clusters of three words or fewer; +- nothing from the "Do not write" list; +- a comment that says why for every block of code. + +Fix each violation before you report the change as done. + ### Never remove When you edit prose, keep: @@ -168,8 +183,34 @@ rules above. Opinionated, project-varying choices belong in `STYLE.local.md`. ## Formatting -- Wrap code and comments at roughly 80 columns; continue long lines with `...`. - (Indentation width is project-specific — see `STYLE.local.md`.) +- Keep code and prose comments within 80 columns; continue code with `...`. + Record lines that cannot be wrapped without changing their meaning or use. +- Indent blocks with 3 spaces. Indent continuations 3 additional spaces from + their statement, without aligning them under an opening bracket. + MATLAB Editor Smart Indent determines indentation; configure it for these widths. + Keep MISS_HIT's indentation rule disabled so it does not undo that layout. +- Put exactly one space on each side of assignment `=` signs. + Remove padding used to align assignments, other code tokens or trailing comments. + Keep one space before a trailing comment and preserve its attachment to the code. + Preserve whitespace that separates MATLAB array elements. +- Use plain section headers such as `%% Header`, without decorative borders. + Keep meaningful comment text and execution cell boundaries. +- Preserve the H1 help convention `%FUNCTION brief description`, without a space + after `%`. Use a space after `%` in ordinary prose comments. + Keep pragmas, block-comment delimiters and commented-out code intact. +- Use the shared MATLAB MISS_HIT profile in a repository-root `miss_hit.cfg`. + Set `project_root`, `line_length: 80`, `tab_width: 3`, and both bracket-alignment + options to `false`. Suppress `operator_after_continuation` to allow leading + continuation operators without moving them for style. + Suppress `whitespace_comments` because MISS_HIT has no H1-specific spacing exception. + Keep the shared profile's name, token and encoding protections during formatting. +- Keep frozen dependencies and project-specific exceptions in local configurations. + Record the formatter path, selected files and justified exceptions in `STYLE.local.md`. + Run `mh_style --fix` only on the selected files, then check Smart Indent and + remove remaining alignment padding. MISS_HIT alone does not enforce every rule above. +- Formatting must preserve behavior, data, RNG order, displayed output, comments + and encoding. Renaming, scalar-operator substitutions, quote rewrites and + scientific changes need their own scope. ## Testing diff --git a/TODO.md b/TODO.md index 63077db..b660094 100644 --- a/TODO.md +++ b/TODO.md @@ -276,6 +276,10 @@ reaches as known external. The rows below record the earlier state. (`globalfit.m:91-93`). `tests/test_corechain.m` requires zero new figures from a global fit with `plotfits` false, and `test_cloudphiPlotfit` covers both flag values. + - update (2026-10-13): `tests/test_corechain.m` is removed. + `TestBaseflow/test_divergentTailHasNoExpectedFlow` holds its check + that a global fit with `plotfits` false opens no figure. This + update supersedes the test_corechain names above. ## Sandbox rehabilitation (bfra-3kh.20 stub, W2) diff --git a/docs/index.html b/docs/index.html index 25d1df0..4073c4d 100644 --- a/docs/index.html +++ b/docs/index.html @@ -9,7 +9,7 @@ Getting Started with the Baseflow Toolbox - +