Skip to content

Correctness fixes, component tiers, and revised plan - #3

Merged
bshepp merged 7 commits into
mainfrom
correctness-fixes
Sep 29, 2026
Merged

bshepp merged 7 commits into
mainfrom
correctness-fixes

Conversation

@bshepp

@bshepp bshepp commented Sep 27, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes the defects found in the September 2026 review, narrows what the project claims to what has been checked, and re-orders the plan.

This branch is stacked on #2 (report withdrawal). Merge #2 first, or merge this one and close #2.

Correctness fixes

Each has a regression test.

Area Defect Fix
BETSE loader Frames loaded as 0, 1, 10, 100, ... because the sort key matched the "2" in "Vmem2D" Frame index anchored to the end of the file name
Topology sublevel and superlevel were swapped Field is negated only for superlevel
Topology Non-square fields passed to GUDHI in the wrong cell order Fortran-order flatten
Topology No-GUDHI fallback counted components at 50 thresholds Exact union-find H0, checked against GUDHI
Topology Wasserstein fallback omitted the power and root, and crashed on infinite bars Rewritten; tested on closed-form cases
Attractors Detectors labelled a sine wave and white noise strange Detectors report UNDETERMINED
Classification Near-zero estimates, including noise, returned FIXED_POINT UNDETERMINED unless the caller asserts oscillatory
Surrogate test Scripts used 30 surrogates, which cannot reach p ≤ 0.05 Raises below the minimum count
Pipeline Failed stages returned success=True success=False, with failed_stages and errors
Pipeline Input array returned as a "reconstruction" Stage reported as skipped
CLI mneme analyze without --config disabled every stage Uses pipeline defaults; exits non-zero on failure
Wiener filter Response rows not normalised; correlation length in the wrong units Both corrected; tested against a known field

Naming

The default reconstructor fits a GP to a random subset of the observations. It is now SubsetGPReconstructor (gp_subset). The dense reconstructor is WienerFilterReconstructor (wiener_filter). Old names still work and emit DeprecationWarning.

Scope

  • docs/SCOPE.md sorts components into core, frozen and experimental.
  • Experimental components emit mneme.ExperimentalWarning.
  • Default pipelines no longer run attractor detection.
  • The Lyapunov modules are frozen. docs/LYAPUNOV_OPERATING_RANGE.md records their measured accuracy and detection power.

Breaking changes

  • classify_attractor(..., oscillatory=...) defaults to None.
  • PersistentHomology(compute_cycles=True) raises NotImplementedError.
  • NeuralFieldReconstructor.uncertainty() raises NotImplementedError.
  • PipelineResult.success is False when any stage fails.
  • BETSE interpolation defaults to linear.
  • Withdrawn PhysioNet result files are removed.

Testing

Local run on Windows, Python 3.13, GUDHI without POT: 372 passed, 1 skipped, coverage 70.4%.

CI now installs GUDHI and POT, so this is the first run of the GUDHI code paths in CI. The coverage floor rises from 35% to 60%.

The largest_lyapunov vectorisation was checked against the original on ten reference signals: identical fit regions, largest difference 2e-14.

Not done

  • The analysis scripts have not been re-run.
  • The multistability experiment has a protocol but no runs. BETSE and the simulation configs are not available in the development environment.

🤖 Generated with Claude Code

bshepp and others added 7 commits September 27, 2026 12:42
The report's time-dependent results were computed on frames loaded out of
time order, and its Lyapunov values came from the removed estimator.
Replace the report with a withdrawal notice, remove it from the site
navigation, and correct the README and docs index claims about chaos
gating and the Wolf algorithm.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ader

The sort key matched the '2' in 'Vmem2D' for every file, so runs of ten or
more frames loaded in directory order (0, 1, 10, 100, ...). Anchor the
frame index to the end of the file stem and raise on unparseable names.

Also add load_betse_cells() for analysis without interpolation, default to
linear interpolation (cubic overshoots the data range), and report an
inside-hull mask and frame indices in the metadata.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- 'sublevel' negated the field before handing it to GUDHI, which already
  computes sublevel persistence, so the two options were swapped.
- Cells were flattened in C order; GUDHI expects the first axis fastest,
  which scrambled non-square fields.
- Replace the threshold-counting fallback with an exact union-find H0
  computation that matches GUDHI, and return empty diagrams for the
  dimensions it does not compute.
- Rewrite the Wasserstein and bottleneck fallbacks (p-th power and root,
  own-diagonal matching only, essential classes ignored). The Wasserstein
  fallback previously crashed on infinite bars.
- All fallbacks now emit RuntimeWarning. NaN fields raise instead of
  yielding empty diagrams. compute_cycles raises NotImplementedError
  instead of returning empty arrays.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…te test

- Detectors in mneme.core.attractors assigned fixed-point, limit-cycle and
  strange labels from variance and spread thresholds, bypassing the
  surrogate gate. A sine wave and white noise were both labelled strange.
  They now report UNDETERMINED.
- classify_attractor returned FIXED_POINT or LIMIT_CYCLE for any near-zero
  estimate, including white noise. Those labels now require the caller to
  assert whether the signal oscillates.
- surrogate_test raises when the surrogate count cannot reach alpha (the
  scripts used n=30, whose smallest p-value is 0.065), and warns on
  multi-dimensional input and on series shorter than 4000 points.
- Recurrence detector counted each time index once per recurrent pixel,
  giving basin sizes above 1.
- Vectorise the divergence loop in largest_lyapunov. Results are unchanged
  to 1e-13 on ten reference signals; runtime drops about tenfold.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The default 'IFT' / 'sparse GP' reconstructor fits an exact GP to a random
subset of the observations and discards the rest. Rename it
SubsetGPReconstructor (method 'gp_subset'). The dense reconstructor is a
Wiener filter; rename it WienerFilterReconstructor ('wiener_filter').
Old class names, method names and n_inducing keep working and emit
DeprecationWarning.

Fixes found while adding ground-truth tests:
- Wiener filter response rows were not normalised, scaling the
  reconstruction down by the number of grid points in a correlation length.
- Wiener filter correlation_length was in pixels but compared with
  unit-square distances.
- optimize_hyperparameters=False still optimised the kernel.
- NeuralFieldReconstructor.uncertainty() returned zeros; it now raises
  NotImplementedError and fit_reconstruct reports uncertainty=None.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- A stage that raised was logged and the run returned success=True. Runs
  now return success=False with the stage in failed_stages and its message
  in errors; the partial analysis result is still returned.
- 'mneme analyze' without --config passed an empty config, which disabled
  every component. The CLI now overlays the user's config on the chosen
  pipeline's defaults, prints each stage's status, and exits non-zero on
  failure.
- Without sparse observations the pipeline returned the input array as a
  'reconstruction'. The stage is now reported as skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Narrow what the project claims to what has been checked.

- Add mneme.ExperimentalWarning. Attractor detectors, symbolic regression,
  the VAE and neural-field reconstruction warn when constructed. Default
  pipelines run core stages only.
- docs/SCOPE.md defines core, frozen and experimental tiers.
- docs/LYAPUNOV_OPERATING_RANGE.md records measured accuracy and detection
  power; the Lyapunov modules are frozen. Probe scripts and their outputs
  are in review_artifacts/2026-09-26.
- docs/MULTISTABILITY_PROTOCOL.md and project_plan.md v3 re-order the work
  around the hypothesis the project is meant to test.
- Comparative summaries report None, not 0, for analyses that did not run.
  The quality checker reports 'unknown' when one of its checks fails.
- Remove withdrawn PhysioNet result files; mark the data acquisition plan
  superseded; update README, CLAUDE.md, CHANGELOG and the API reference.
- CI installs GUDHI and POT, and the coverage floor rises from 35% to 60%.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@bshepp
bshepp merged commit dccec23 into main Sep 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant