Skip to content

feat(core): add shared source-tree views and dependency tracking - #775

Draft
frederikgeth wants to merge 1 commit into
mainfrom
codex/source-tree-views
Draft

frederikgeth wants to merge 1 commit into
mainfrom
codex/source-tree-views

Conversation

@frederikgeth

Copy link
Copy Markdown
Collaborator

What and why

Part of #774 — P1.

Several case entry points can reuse topology and data without rebuilding the supplied file inventory or sharing parser state. This adds SourceTree and SourceTreeLimits in powerio-core, re-exported by the facade. Each selection exposes a directory or a primary file relative to a selected root, sharing immutable bytes while rebasing names, source IDs, and reference directories.

Explicit limits bound exposed file count, combined byte length, and path depth; they do not read environment variables. Source::dependencies() reports successful nonprimary reads in deterministic order. Source clones share observations, while independent selections do not. Listing or replaying retained companions and failed reads do not add dependencies. Retained companions remain available for byte-exact replay.

Tests cover shared byte identity, root confinement, foreign-referrer rejection, exact limit boundaries, independent observations, two OpenDSS masters parsed in either order, legacy-parser value/diagnostic parity, source replay, and an isolated PyPSA directory view.

Decisions for the maintainer

None outstanding beyond review of the additive P1 API. The implementation keeps the existing 64-segment resolver bound, case-sensitive paths, and explicit view limits. A view retains its shared tree owner; its smaller exposure limit does not imply that the rest of the shared allocation is released.

Not in this pull request

ZIP decoding, case discovery, document-role grouping, collection manifests, CLI/browser selection, and binding entry points remain follow-ups in #774. This changes no electrical interpretation, C ABI, or PowerIO IR layout.

Files outside the main change

  • powerio-core/src/lib.rs and powerio/src/lib.rs: public re-exports.
  • CHANGELOG.md: the additive source API under Unreleased.
  • powerio/tests/source_tree_views.rs: facade-level parsing and exact-replay regressions alongside the core source tests.

No dependency, workflow, generated header, schema snapshot, or conversion baseline changes.

Fixtures

No new fixture files. Test inputs are small synthetic byte/string literals.

Checks

Commands used CARGO_TARGET_DIR=/private/tmp/powerio-source-tree-target.

Passed:

  • cargo fmt --all --check and git diff --check.
  • cargo test: 3,413 passed across the default workspace and doctests.
  • cargo test -p powerio-capi: 54 passed, one existing ignored test.
  • cargo test -p powerio-core: rerun after review, including the new integration tests and rustdoc example.
  • cargo test -p powerio --test source_tree_views --test memory_tree --no-default-features.
  • cargo clippy -p powerio-core --all-targets -- -D warnings.
  • cargo clippy -p powerio --lib --test source_tree_views --no-default-features -- -D warnings.
  • cargo check -p powerio --no-default-features --target wasm32-unknown-unknown.
  • RUSTDOCFLAGS="-D warnings" cargo doc -p powerio-core -p powerio --no-deps --all-features.

Existing Clippy failures

bash scripts/ci-clippy.sh was rerun with rustc 1.96.0 / Clippy 0.1.96. The workspace target fails on existing clippy::float_cmp assertions:

Both files are byte-identical to upstream main at 8335e3b, verified by comparing their Git blob IDs. Neither file nor the lint policy is changed here.

Because the full script stops after its first failed target, the remaining configurations were also run individually:

Target Result
workspace Fails on the 17 PSS/E and 3 matrix assertions above
matrix-gridfm Fails on the same 3 matrix assertions
no-sincal Fails on the same 17 PSS/E assertions
capi-no-default Pass
capi-release Pass
capi-dist Pass
powerio-py Pass

The PR remains draft while the full Clippy gate is red; it is not asking for review yet. No unrelated baseline fixes are bundled into P1.

Not run: scripts/ci-mirror.sh and mdBook/SemVer checks. The local mdBook and cargo-semver-checks executables are unavailable, and the full Clippy gate has the failures documented above. Cross-platform, MSRV, Julia, fuzz, and independent format-oracle jobs are left to CI; this PR adds source acquisition APIs without changing electrical parser semantics.

@github-actions

Copy link
Copy Markdown
Contributor

Conversion Matrix

Each cell writes the source as the target and reads it back. The symbol is the worst finding.

⚪ exact 🟢 lossless 🔵 restated: only values the target requires were synthesized

🟡 approximate: values changed to fit 🟠 dropped: source data has no target field

🔴 failed: error, invariant, or baseline change

Transmission: into a case format

From ↓ / to → Source MATPOWER .m PowerModels JSON PSS/E .raw 33 PSS/E RAWX 35 PowerWorld .aux egret JSON pandapower JSON Surge JSON PSLF .epc
MATPOWER .m 🟢 ⚪ 🟠 1 areas 🟠 16 field 🟠 16 field 🟠 12 field 🟠 7 field 🟠 12 field 🟠 23 field 🟠 14 field
PowerModels JSON 🟢 🟢 ⚪ 🟠 16 field 🟠 16 field 🟠 11 field 🟠 6 field 🟠 11 field 🟠 22 field 🟠 13 field
PSS/E .raw 33 🟢 🟠 1 field 🟠 1 areas ⚪ 🟢 🟠 2 areas 🟠 1 areas 🟠 2 areas 🟠 1 areas 🟠 2 areas
PSS/E RAWX 35 🟢 🟠 1 field 🟠 1 areas 🟢 ⚪ 🟠 2 areas 🟠 1 areas 🟠 2 areas 🟠 1 areas 🟠 2 areas
PowerWorld .aux 🟢 🟠 1 field 🟢 🟠 6 field 🟠 6 field ⚪ 🟢 🟠 1 field 🟢 🟠 6 field
egret JSON 🟢 🟢 🟢 🟠 10 field 🟠 10 field 🟠 6 field ⚪ 🟠 5 field 🟠 1 field 🟠 7 field
pandapower JSON 🟢 🟢 🟢 🟠 8 field 🟠 8 field 🟠 2 field 🟢 ⚪ 🟢 🟠 7 field
Surge JSON 🟢 🟢 🟢 🟠 10 field 🟠 10 field 🟠 6 field 🟢 🟠 4 field ⚪ 🟠 7 field
PSLF .epc 🟢 🟢 🟢 🟠 1 field 🟠 1 field 🟠 1 record 🟢 🟠 3 field 🟠 3 field ⚪
XIIDM 1.17 🟢 🟠 7 field 🟠 7 field 🟠 38 record 🟠 38 record 🟠 8 field 🟠 7 field 🟠 8 field 🟠 7 field 🟠 8 field
JIIDM 1.17 🟢 🟠 7 field 🟠 7 field 🟠 38 record 🟠 38 record 🟠 8 field 🟠 7 field 🟠 8 field 🟠 7 field 🟠 8 field
CGMES 3.0 🟢 🟠 6 field 🟠 6 field 🟠 73 field 🟠 42 field 🟠 6 field 🟠 6 field 🟠 7 field 🟠 6 field 🟠 6 field
UCTE-DEF .uct 🟢 🟠 18 field 🟠 12 areas 🟠 6 extras 🟠 6 extras 🟠 18 areas 🟠 12 areas 🟠 6 areas 🟠 12 areas 🟠 18 areas
PyPSA CSV 🟢 🟢 🟡 6 nonfinite 🟠 8 field 🟠 8 field 🟠 2 field 🟡 6 nonfinite 🟠 2 field 🟡 6 nonfinite 🟠 6 field
GridFM Parquet 🟢 🟠 6 field 🟢 🟠 14 field 🟠 14 field 🟠 8 field 🟠 6 field 🟠 8 field 🟢 🟠 12 field
PSS/E .raw 32 🟠 2 section 🟠 2 field 🟠 2 areas 🟢 🟢 🟠 2 areas 🟠 2 areas 🟠 2 areas 🟠 2 areas 🟠 4 field
IEEE CDF 🟡 2 value 🟠 2 field 🟠 2 areas 🟠 1 field 🟠 1 field 🟠 2 areas 🟠 2 areas 🟠 2 areas 🟠 2 areas 🟠 2 areas
GO Challenge 3 JSON 🟡 1 value 🟠 5 rating set 🟠 4 rating set 🟠 6 rating set 🟠 3 field 🟠 8 field 🟠 4 rating set 🟠 6 field 🟠 4 rating set 🟠 6 rating set
DeepMind OPFData JSON 🟠 1 source only 🟠 1 field 🟢 🟠 3 field 🟠 3 field 🟠 2 field 🟠 1 field 🟠 2 field 🟢 🟠 2 field

Transmission: into an exchange format

From ↓ / to → Source XIIDM 1.17 JIIDM 1.17 CGMES 3.0 UCTE-DEF .uct
MATPOWER .m 🟢 🟠 38 field 🟠 38 field 🟠 50 field 🟠 28 field
PowerModels JSON 🟢 🟠 38 field 🟠 38 field 🟠 49 field 🟠 27 field
PSS/E .raw 33 🟢 🟡 1 value 🟡 1 value 🟠 5 record 🟠 15 field
PSS/E RAWX 35 🟢 🟡 1 value 🟡 1 value 🟠 5 record 🟠 15 field
PowerWorld .aux 🟢 🟠 23 field 🟠 23 field 🟠 23 field 🟠 19 field
egret JSON 🟢 🟠 37 field 🟠 37 field 🟠 29 field 🟠 22 field
pandapower JSON 🟢 🟠 25 field 🟠 25 field 🟠 29 field 🟠 18 field
Surge JSON 🟢 🟠 36 field 🟠 36 field 🟠 29 field 🟠 22 field
PSLF .epc 🟢 🟠 7 field 🟠 7 field 🟠 4 record 🟠 14 field
XIIDM 1.17 🟢 ⚪ 🟢 🟠 179 field 🟠 21 field
JIIDM 1.17 🟢 🟢 ⚪ 🟠 179 field 🟠 21 field
CGMES 3.0 🟢 🟠 19 field 🟠 19 field ⚪ 🟠 17 field
UCTE-DEF .uct 🟢 🟠 6 field 🟠 6 field 🟠 6 record ⚪
PyPSA CSV 🟢 🟠 33 field 🟠 33 field 🟠 25 field 🟠 19 field
GridFM Parquet 🟢 🟠 33 field 🟠 33 field 🟠 25 field 🟠 26 field
PSS/E .raw 32 🟠 2 section 🟡 4 value 🟡 4 value 🟠 6 field 🟠 6 field
IEEE CDF 🟡 2 value 🟡 1 value 🟡 1 value 🟠 1 record 🟠 5 field
GO Challenge 3 JSON 🟡 1 value 🟠 2 field 🟠 2 field 🟠 5 rating set 🟠 10 field
DeepMind OPFData JSON 🟠 1 source only 🟠 25 field 🟠 25 field 🟠 5 field 🟠 7 field

Transmission: into a dataset directory

From ↓ / to → Source PyPSA CSV GridFM Parquet
MATPOWER .m 🟢 🟠 15 field 🟠 18 field
PowerModels JSON 🟢 🟠 14 field 🟠 17 field
PSS/E .raw 33 🟢 🟠 9 field 🟠 13 field
PSS/E RAWX 35 🟢 🟠 9 field 🟠 13 field
PowerWorld .aux 🟢 🟠 7 field 🟠 11 field
egret JSON 🟢 🟠 9 field 🟠 12 field
pandapower JSON 🟢 🟠 8 field 🟠 7 field
Surge JSON 🟢 🟠 9 field 🟠 12 field
PSLF .epc 🟢 🟠 8 field 🟠 12 field
XIIDM 1.17 🟢 🟠 15 field 🟠 39 field
JIIDM 1.17 🟢 🟠 15 field 🟠 39 field
CGMES 3.0 🟢 🟠 13 field 🟠 37 field
UCTE-DEF .uct 🟢 🟠 12 areas 🟠 24 field
PyPSA CSV 🟢 ⚪ 🟠 6 field
GridFM Parquet 🟢 🟠 12 field ⚪
PSS/E .raw 32 🟠 2 section 🟠 4 field 🟠 6 field
IEEE CDF 🟡 2 value 🟠 3 field 🟠 3 field
GO Challenge 3 JSON 🟡 1 value 🟠 8 field 🟠 7 field
DeepMind OPFData JSON 🟠 1 source only 🟠 3 field 🟠 2 field

Distribution: into a distribution format

From ↓ / to → Source OpenDSS .dss BMOPF JSON PMD JSON
OpenDSS .dss 🟠 11 source only ⚪ 🟠 81 field 🟠 188 field
BMOPF JSON 🟢 🟠 31 field ⚪ 🟠 11 field
PMD JSON 🟢 🟠 47 extras 🟠 45 field ⚪

Full warning details: download conversion-matrix-warning-details for this workflow run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant