Fix pipeline state, evaluation consistency, and geometry exports - #64
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Legacy PLS migration leaves saved CLOISTER geometry misaligned, and serial SIFTED still dispatches work to existing parallel pools.
Review effort: Balanced
Findings: 2
Open (4)
What changed in this PR
This PR corrects pipeline state, evaluation consistency, scientific metrics, geometry exports, and performance bottlenecks, with regression coverage and regenerated documentation.
Changes:
- Fixes preprocessing, staged rebuilds, PLS projection, selection metrics, TRACE evaluation, and migration.
- Improves geometry exports, plotting, parallel-pool handling, and memory usage.
- Adds extensive regression tests, verification evidence, and reference documentation.
| File | Description |
|---|---|
utils/ISAvalidateOpts.m |
Canonicalizes options and validates seeds/subsetting. |
utils/ISAmigrateModel.m |
Migrates PLS and PYTHIA state. |
utils/ISAdefaults.m |
Adds SIFTED diagnostics default. |
tests/ValidationTest.m |
Updates option-validation expectations. |
tests/StateReviewTest.m |
Tests state, rebuild, path, and pool behavior. |
tests/StageUnitTest.m |
Tests cache isolation and camera angles. |
tests/ReviewFixTest.m |
Covers reviewed pipeline defects. |
tests/ProjectionReviewTest.m |
Tests projection fallback and allocation behavior. |
tests/PortfolioPruningTest.m |
Verifies preprocessing after portfolio pruning. |
tests/PerformanceReviewTest.m |
Tests bounded hull and filtering behavior. |
tests/OptionsReviewTest.m |
Tests JSON groups and camera direction. |
tests/NumericReviewTest.m |
Tests performance-domain and seed boundaries. |
tests/InstanceSpaceTest.m |
Checks stale plot removal. |
tests/ExportReviewTest.m |
Verifies 2D/3D geometry exports. |
review/2026-09-30/smoke.log |
Records smoke-test output. |
review/2026-09-30/run_review_verification.m |
Runs partitioned verification suites. |
review/2026-09-30/reproductions.log |
Records original defect reproductions. |
review/2026-09-30/REPORT.md |
Documents initial review findings. |
review/2026-09-30/isa_review_smoke.m |
Defines the review smoke test. |
review/2026-09-30/isa_review_extra.m |
Defines additional reproductions. |
review/2026-09-30/isa_review_checks.m |
Defines primary defect reproductions. |
review/2026-09-30/FINAL_REPORT.md |
Summarizes resolutions and validation. |
review/2026-09-30/check_svm_calibration_labels.m |
Checks SVM calibration effects. |
README.md |
Updates missing-data and TRACE contracts. |
output/scriptpng.m |
Removes stale plots and adjusts performance scaling. |
output/scriptfcn.m |
Corrects 3D camera conventions. |
output/scriptcsv.m |
Exports 3D faces and geometry manifests. |
output/ISArecallView.m |
Restores equal-axis camera views. |
InstanceSpace.m |
Fixes state provenance, pruning, pools, and PLS evaluation. |
doc/src/TRACE.md |
Documents evaluation geometry and acceptance status. |
doc/src/SIFTED.md |
Documents optional diagnostics. |
doc/src/scriptpng.md |
Documents stale plot replacement. |
doc/src/scriptcsv.md |
Documents geometry export schema. |
doc/src/PYTHIA.md |
Documents metrics, scores, and selection changes. |
doc/src/PRELIM.md |
Documents performance-domain behavior. |
doc/src/PILOTviewpoint.md |
Documents camera conventions. |
doc/src/PILOT.md |
Documents PLS centering and optimizations. |
doc/src/OptionsReference.md |
Updates option contracts. |
doc/src/MetadataFormat.md |
Clarifies missing-value handling. |
doc/src/InstanceSpace.md |
Documents stage provenance and rebuilding. |
doc/src/GettingStarted.md |
States performance-domain requirements. |
doc/src/FILTER.md |
Documents bounded-neighborhood processing. |
doc/src/CLOISTER.md |
Documents centering and batched corners. |
doc/html/TRACE.html |
Regenerates TRACE reference output. |
doc/html/SIFTED.html |
Regenerates SIFTED reference output. |
doc/html/scriptpng.html |
Regenerates plot-export documentation. |
doc/html/scriptcsv.html |
Regenerates CSV-export documentation. |
doc/html/PYTHIA.html |
Regenerates PYTHIA documentation. |
doc/html/PRELIM.html |
Regenerates PRELIM documentation. |
doc/html/PILOTviewpoint.html |
Regenerates viewpoint documentation. |
doc/html/PILOT.html |
Regenerates PILOT documentation. |
doc/html/OptionsReference.html |
Regenerates options documentation. |
doc/html/MetadataFormat.html |
Regenerates metadata documentation. |
doc/html/InstanceSpace.html |
Regenerates class documentation. |
doc/html/GettingStarted.html |
Regenerates getting-started documentation. |
doc/html/FILTER.html |
Regenerates FILTER documentation. |
doc/html/CLOISTER.html |
Regenerates CLOISTER documentation. |
core/TRACE.m |
Reuses trained geometry and exposes acceptance state. |
core/SIFTED.m |
Resets caches and controls diagnostics/parallelism. |
core/PRELIM.m |
Fixes performance-domain and normalization behavior. |
core/PILOTviewpoint.m |
Respects serial execution settings. |
core/PILOT.m |
Applies PLS centering and avoids unnecessary distances. |
core/INIT.m |
Replays feature exclusions and rejects incomplete training data. |
core/FILTER.m |
Bounds neighborhood allocation. |
core/CLOISTER.m |
Centers PLS bounds and batches corner enumeration. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #64 +/- ##
==========================================
+ Coverage 94.18% 94.88% +0.69%
==========================================
Files 23 23
Lines 3045 3282 +237
==========================================
+ Hits 2868 3114 +246
+ Misses 177 168 -9 ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeated builds and evaluation could reuse stale state, inconsistent preprocessing, or test-derived selection weights. This change corrects those paths and adds regression coverage for the reviewed scientific and export contracts.
Changes
Validation
See the resolution report and per-test evidence for finding-to-commit mappings, exact validation scope, and limitations. Historical reproduction failures are retained as review evidence.
Compatibility
Rebuild affected experiments to obtain corrected preprocessing and summaries. Incomplete training data and negative raw performance now raise errors. Relative zero handling remains a unit-sensitive numerical convention. Training CV remains conditional on the fitted pipeline rather than an unbiased outer-CV assessment. Saved pre-SIFTED data increases model size.
Copilot follow-up
Commit
f238f16addresses all four findings: serial SIFTED no longer dispatches worker cache resets, legacy PLS migration translates saved CLOISTER vertices once, and CLOISTER/PYTHIA help now documents the centring argument and score semantics. All 12 targeted tests passed, including serial/parallel cache isolation and 2D/3D migration with unchanged topology. Documentation generation and all 31 page link checks passed. See the follow-up report and test results.Issue #63 investigation
This PR should not be considered a fix for #63. A controlled comparison of master and the PR ran 12 PILOT solves in six MATLAB R2026a Update 5 sessions on identical saved inputs, with default (12) and single computation threads. All runs selected restart 10. Coefficients and projected coordinates were exactly equal; switching thread count changed only restart correlation scores by at most 6.55e-15. The reported larger CI drift was not reproduced. The numerical objective, optimization loop, and raw restart-selection rule remain unchanged, so #63 remains open. See the investigation and reproduction instructions.
A Balanced Copilot re-review was requested after fixing the four original findings, but Copilot could not run it because the requesting account reached its quota limit.