tree: Replace ExternalNoVerity with ExternalPath, keep ostree's redirect - #409
Open
cgwalters-bot wants to merge 1 commit into
Open
cgwalters-bot wants to merge 1 commit into
cgwalters-bot wants to merge 1 commit into
Conversation
ostree gives each file a libcomposefs payload of `xx/<checksum>.file` and sets the fsverity digest separately. The capi turned the payload into an ObjectID (dropping `.file`) or ignored it when a digest was set, so the redirect pointed at a file that doesn't exist and an ostree deployment on the Rust libcomposefs couldn't boot. ExternalPath carries an optional redirect as given plus an optional verity digest, which is how libcomposefs models it. RegularFile::external() builds it from the pair and keeps the usual composefs layout (redirect is the digest's object path) as External, so the capi, the EROFS reader and the dumpfile parser all normalize the same way. A digest without a payload now stays that way everywhere, as in C: no redirect is made up for it, and the dumpfile parser accepts it. composefs-info lists the redirect as the object path, like C's, so ostree images don't lose their objects there. In V2 images ExternalPath gets the same single null chunk index as External. Note Item::Regular's path in dumpfile_parse is now an Option, and a redirect with an interior NUL fails the capi conversion instead of being dropped. The new test builds an image like ostree does via the C API and pins its digest; `just test-capi` compares the same image against the C library. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
cgwalters
requested changes
Oct 2, 2026
| Self::External(id, _) => Ok(id.clone()), | ||
| Self::ExternalPath { | ||
| verity: Some(id), .. | ||
| } => Ok(id.clone()), |
Collaborator
There was a problem hiding this comment.
Couldn't this function return a borrowed value?
Comment on lines
+280
to
+281
| let start = bytes.iter().position(|&b| b != b'/').unwrap_or(bytes.len()); | ||
| Path::new(OsStr::from_bytes(&bytes[start..])) |
| use zerocopy::IntoBytes; | ||
|
|
||
| let name = CString::new("ostree-image").unwrap(); | ||
| let fd = unsafe { libc::memfd_create(name.as_ptr(), 0) }; |
Collaborator
There was a problem hiding this comment.
There's rustix APIs for this ensure need a review checklist item to prefer rustix over libc
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ostree gives each file a libcomposefs payload of
xx/<checksum>.fileand sets the fsverity digest separately. composefs-capi turned the payload into anObjectID(dropping.file), or ignored it when a digest was set, so the overlay redirect pointed at a file that doesn't exist and an ostree deployment on the Rust libcomposefs couldn't boot ("overlayfs: lazy lowerdata lookup failed", "No /sbin/init").This replaces
RegularFile::ExternalNoVeritywithExternalPath { redirect: Option<_>, verity: Option<_>, size }, which is how libcomposefs models a file: an optional redirect as given, plus an independent optional digest for the metacopy xattr.RegularFile::external()builds one from that pair and normalizes it, so the capi, the EROFS reader and the dumpfile parser all agree: a redirect that is the digest's object path givesExternal(the usual composefs layout), neither givesSparse, and anything else isExternalPath. The capi maps payload and digest exactly like C (empty files get no metacopy or redirect, an empty payload writes no redirect).RegularFile::backing_path()gives the redirect target of either external variant, whichcomposefs-info ls/objects/missing-objectsandcfsctl's backing-path dump now list, like C'scomposefs-infodoes with the payload.RegularFile::repo_object_id()gives callers that open the backing object from a repository the object for either variant.This is a public API change, and so is
dumpfile_parse::Item::Regular'spathbecoming anOption(see below). bootc needs a small change for both (bot/composefs-externalpathon cgwalters-bot/bootc, on top ofbootc-dev/bootc#2490): fourExternalNoVeritymatches and oneItem::Regularin its GC. That only matters when bootc bumps to a composefs-rs release with this; the revdep workflow is disabled until then anyway.Tested on a 16-core RHEL 10 devspace (kernel 6.12):
OSTREE_IMAGE_C_DIGESTcomes from the C libcomposefs (composefs/composefsec2573a):tests/ostree-image.cbuilt against it in a Fedora container, thenfsverity digestof its output. Besides ostree's forms (.filewith and without a digest, an empty file, a symlink), the image has a digest-only file and one whose payload is the digest's object path. The capi testtest_ostree_image_matches_cbuilds the same image through our capi, checks what the reader makes of each file, and gets the same digest.just test-capipasses, including the step thatcmps the image fromostree-image.cwritten by each library, and the Ctest-checksums.sh/test-units.shsuites.cargo test --workspace --exclude composefs-integration-tests -- --skip fsverity::in a Fedora container like thefedoraCI job: all pass (composefs 219, capi 22). After the last round of fixes,cargo test -p composefs-capi -p composefs-ctl,cargo clippy --workspace -- -D warnings, fmt andjust test-capiwere rerun and pass. The fsverity tests were skipped since the container's /var/tmp has no fs-verity.just clippy,check-feature-combos,fmt-checkandcheck-fuzzpass.just test-integrationin the same container: 109 passed, 3 failed, and the same 3 fail on main there (no skopeo, nosecurity.selinuxon the container's /var/tmp, and a varlink connection reset).just bootc/test-ostree(bootc from the branch above) builds the composefs RPMs, confirms libostree and the initramfs load our libcomposefs, and bootc'sreadonlyandimage-upgrade-rebootplans on the ostree backend pass, so the deployment boots and upgrades through the Rust libcomposefs.Design notes:
ExternalPath { redirect: None, verity: Some(_) }everywhere now: the capi no longer makes up a redirect from the digest, the reader no longer turns such a file intoExternal(which would add a redirect when the image is rewritten), and the dumpfile parser accepts payload-with a digest, as C does and as our own writer emits. For that,Item::Regular'spathis now anOption. Such a file has no backing path, socomposefs-info objectsdoesn't list it, like C.lcfs_load_node_from_image*, with the error logged at debug) instead of silently dropping the redirect. The test for it has a subdirectory and a hardlinked pair converted before the bad file, so freeing the partial tree is exercised; it passes under ASan (nightly-Zsanitizer=address, with LeakSanitizer) on the devspace.composefs-info missing-objectstrims leading slashes before joining a payload to--basedir, like C'sabs_to_rel_path(), so a/-prefixed payload can't escape it.Externalgets.ExternalNoVeritygot none there. That was harmless in practice, since only the capi produced it and the capi writes V1.ExternalPathgets the same index asExternal. V1 output is unchanged, and no pinned V2 digest moved.Sparsein V2 still gets no chunk index; that looks like the same issue but is left alone here.Related: #323
The
Signed-off-by: Colin Walters <walters@verbum.org>on these commits was added on cgwalters's approval of the review draft: cgwalters-forge#7 (review)Generated-by: https://github.com/cgwalters/#llms