Repository navigation
feat(archive): add bounded model-neutral ZIP transport - #776
Draft
frederikgeth wants to merge 1 commit into
Draft
frederikgeth wants to merge 1 commit into
frederikgeth wants to merge 1 commit into
Conversation
Contributor
Conversion MatrixEach cell writes the source as the target and reads it back. The symbol is the worst finding. ⚪ exact 🟢 lossless 🔵 restated: only values the target requires were synthesized 🟡 approximate: values changed to fit 🟠 dropped: source data has no target field 🔴 failed: error, invariant, or baseline change Transmission: into a case format
Transmission: into an exchange format
Transmission: into a dataset directory
Distribution: into a distribution format
Full warning details: download |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Part of #774 (P2). Add
powerio-archive, a model-neutral, in-memory ZIP backend that lower adapters and collection ingestion can share. This PR targetsmaindirectly and is independent of P1 (#775).Opening validates bounded ZIP32/ZIP64 metadata, exact UTF-8 names, portable namespace/collision rules, local headers, streaming descriptors and nonoverlapping member regions. Reads enforce declared size and CRC before publishing shared cached bytes. Defaults are 64 MiB input, 4096 entries, 64 MiB total declared expansion and a 200:1 member ratio; callers supply explicit overrides and optional smaller read caps. Original container bytes remain available. Indexing alone does not certify unread payload CRCs.
Metadata preflight precedes the ZIP decoder's index allocations. During decoder indexing, member regions are masked so embedded end signatures cannot drive fallback metadata parsing; original bytes are exposed again before local-header validation and reads. Collision indexing stores segments with parent IDs instead of copying every ancestor prefix. A one-off allocation probe with 128 disjoint 64-segment paths (994,710 input bytes) reduced peak indexing allocations from 34,449,437 to 3,612,528 bytes.
The transport and its admission tests form one review unit (~1.2k Rust lines). Suggested review order:
preflight.rs,local.rs, the namespace/cache API inlib.rs, then adversarial tests.Decisions for the maintainer
None. This is a draft and is not requesting maintainer review yet, respecting the existing review queue.
Not in this pull request
Electrical interpretation, automatic case discovery, source-tree integration, recursive ZIP traversal, CLI/bindings, and changes to existing parser entry points. P3 will evaluate reuse in CGMES/SINCAL while preserving their compatibility contracts; collection selection follows separately. Nested payloads remain opaque. These limits do not bound downstream decoded tables/models or repeated parser work.
Files outside the main change
Cargo.tomlandCargo.lock: workspace/default membership and the new crate; reuse the existing ZIP dependency without upgrading dependencies.scripts/ci-clippy.sh,ci-mirror.sh,check-release-versions.sh,check-semver.sh,release-verified.py: test, document, package and publish the new crate after core; skip comparison only when the selected SemVer baseline predates the crate.powerio/tests/crate_graph.rs: enforce that archive transport depends on core without electrical models.docs/diagrams/architecture.dotand generateddocs/src/assets/architecture.svg: add its dependency edge; regenerated withpython3 scripts/check-architecture-map.py --render.Fixtures
No new fixture files. Small archives are generated in Rust tests, including malformed metadata/payloads, namespace conflicts, overlap, streaming descriptors, ZIP64 and systematic bit mutations. Independent Python
zipfileprobes cover eight stored/deflated, seekable/streaming, ordinary/forced-ZIP64 combinations; those temporary inputs are not committed.Checks
Passed locally:
cargo fmt --all --checkcargo testandcargo test -p powerio-capicargo test -p powerio-archive(12 tests)cargo clippy -p powerio-archive --all-targets -- -D warningscargo check -p powerio-archive --target wasm32-unknown-unknownRUSTDOCFLAGS="-D warnings" cargo doc -p powerio-archive --no-depscargo package -p powerio-archive --allow-dirty --offline, plusscripts/audit-release-archives.pyon the resulting packageExisting Clippy failures, on rustc 1.96.0 / Clippy 0.1.96:
bash scripts/ci-clippy.shstops in its workspace target on 17clippy::float_cmpfindings inpowerio-tx/src/format/psse.rs: 13269–13274, 13287–13288, 13313–13321. The separately runno-sincaltarget has the same failures.matrix-gridfmtarget has threefloat_cmpfindings inpowerio-matrix/tests/synth_facade_parity.rs: 25, 39, 40.origin/mainat8335e3bf85a158a05e3eeada372c7890ef6e7edc. They are outside this diff. The C ABI configurations and PyO3 Clippy target pass.bash scripts/ci-mirror.shwas attempted and stops at the existing workspace Clippy failures, so its later stages were not reached; the checks listed above ran separately.cargo-semver-checksandmdbookare unavailable locally. Remaining platform/MSRV and integration gates are left to CI.