[FEAT] Migrate unsigned sidecars through the signed write path and delete the second reader (S-D24, S-D23 owed) - #455
Open
justin13888 wants to merge 3 commits into
Conversation
…-unsigned-migration-412
A catalog stamped by a newer build was refused correctly by the migrator,
but `DatabaseDriver::open` flattened `MigrationError::CatalogTooNew` into a
`SqliteFailure` message, `open_library` re-wrapped that as `LibraryError::Db`,
and `Workspace::open` stringified it into `LifecycleError::Io`, so a client
could only print it. This is the owed half of S-D23.
`open_library` now goes through a crate-private `DatabaseDriver::open_typed`
and returns `LibraryError::CatalogTooNew { found, supported }`, with the
catalog untouched and the lock released; `Workspace::open` surfaces every
library-open failure as the new `LifecycleError::Library(LibraryError)`.
The public `DatabaseDriver::open`, consumed by capsule-core-ffi, keeps
flattening, so its signature and behaviour are unchanged.
Deploying capsule with
|
| Latest commit: |
8fa1713
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://cb93a1f4.capsule-22k.pages.dev |
| Branch Preview URL: | https://feat-sidecar-unsigned-migrat.capsule-22k.pages.dev |
A library written before the signed path holds, beside each original, a
flat unsigned CBOR sidecar with no provenance chain, no sealed metadata
blob, and no album key. `Workspace::open` anchors on chains, so such an
asset was silently invisible: it could not be verified, exported, or
uploaded, while a keyless `rebuild_index` still indexed it (S-D24).
`Workspace::migrate_unsigned_sidecars(&UnsignedMigrationOptions)` is the
explicit verb that brings such a library forward. Per legacy record, in
asset-id order, it refuses without writing (bad name, id collision, missing
or hash-mismatched original, a conflicting quarantine twin), copies the
legacy bytes verbatim to `.library/quarantine/{uuid}.cbor` with a sibling
`.reason.json`, then admits the asset as a signed create through the one
create commit every import takes — keeping the legacy id, the media
bucket (nothing moves; the original is signed where it lies), the legacy
import time, and folding the whole legacy map into the signed sidecar's
`_unknown` under `legacy-unsigned-sidecar`, where the signature covers it.
Rating, tags, GPS and capture time land in their signed registers with
the import precedence (EXIF over the legacy record over the legacy import
time, never now); `is_deleted` becomes a signed `delete`; `stack_hint`
groups of two or more get a deterministic v8 stack id over the user id
and the group key, written at create. The run ends with `rebuild_index`.
It is idempotent and resumable: an interrupted run's chainless or torn
sidecar is redone from its quarantine copy, but only while it is still the
migration's own create; an owed `delete` is applied on the next run unless
the asset has since been restored by hand.
Supporting pieces: a crate-private `sidecar::shape::probe` tells the two
shapes apart by their two discriminating keys alone (integer key 0 vs the
text key `version`) and builds no model of either; `Workspace::open` now
records every sidecar no chain anchors and exposes it through
`unmigrated_sidecars()`, warning once per file and naming the verb, so the
library still opens; and the body of `import_asset_with` is extracted into
`commit_signed_create(&CreateRequest)` with no change to the public
signature or to an ordinary import's behaviour — the migration is the one
caller that pins an id, a bucket, an import time, and a fold.
The unsigned reader in `rebuild_index` is left in place for the commit
that deletes it; `DatabaseDriver::open`'s doc no longer links the
crate-private `open_typed` from a public item.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Retires the unsigned pre-signed-path sidecar shape (
AssetSidecar) by migrating every such sidecar into a signedSidecarV1+ manifest + provenance chain through the existing lifecycle write path, then deletes the unsigned reader solibrary::rebuild_indexhas one shape to read (S-D24). Closes S-D23's owed item: a catalog stamped by a newer build is a typed refusal at theopenboundary.Stacked on
feat/media-rawshift-still-decode-410(PR #436); the PR targets that branch.Summary
LibraryError::CatalogTooNew { found, supported }fromopen_libraryvia a crate-privateDatabaseDriver::open_typed(the publicopenkeeps flattening forcapsule-core-ffi);Workspace::opensurfaces every library-open failure as the newLifecycleError::Library(LibraryError)instead of anIostring. Catalog untouched, lock released, tested at both boundaries.sidecar::shape::probe(crate-private) classifies a{uuid}.cborby its two discriminating keys alone — integer key0(signed) vs textversion(legacy) — building no model of either shape.Workspace::openstill succeeds on a library holding unsigned sidecars; a second pass records every sidecar no.provenance.cboranchors and exposes it throughWorkspace::unmigrated_sidecars()(UnmigratedSidecar { path, asset_id, shape }), onewarnper file naming the verb.Workspace::migrate_unsigned_sidecars(&UnsignedMigrationOptions { fallback_album, trash_retain_days }) -> UnsignedMigrationReport. Per legacy record, in asset-id order: refuse without writing (MigrationSkip::{InvalidAssetId, UnknownShape, Undecodable, IdCollision, OriginalMissing, HashMismatch, QuarantineConflict, Stranded}); copy the legacy bytes verbatim to.library/quarantine/{uuid}.cbor+{uuid}.reason.json; admit the asset as a signed create with its legacy id, in its legacy bucket, bytes signed in place, through the create commit extracted fromimport_asset_with(commit_signed_create(&CreateRequest), no public signature change); carryis_deletedas a signeddelete; derive UUIDv5 stack ids forstack_hintgroups of two or more and write them at create. Ends by callingrebuild_index. Idempotent; resumes an interrupted run from the quarantine copy._unknownunderlegacy-unsigned-sidecar, covered by the signature.sidecar::{AssetSidecar, StackHint, read_sidecar}(+cfg(test)write_sidecar),sidecar/{asset_sidecar.rs, stack_hint.rs}, the legacy branch, projection, stack rebuild, helpers, and tests oflibrary/rebuild.rs;capsule-core-ffi::{AssetSidecarRecord, StackHintRecord, serialize_sidecar, deserialize_sidecar}andcapsule-core-ffi/src/sidecar.rs; the SwiftFFISidecarCoder, its test file, and theCatalogFFIBridgeextensions over the deleted records.design/import/pipeline.mdstatus note, adesign/filesystem/maintenance.mdrepair-table row, SLICES rows and detail blocks for S-D23 and S-D24.SLICES counts delta (not applied — the counts paragraph is owned by another lane): status
done+1,blocked−1 (S-D24blocked→done); the "Seven rows readblocked" paragraph and theD21 --> D24mermaid edge atSLICES.md:419-421, :436also describe S-D24 as blocked and are left for the same owner.Validation
Slice 1 (typed catalog-too-new), inside the worktree:
cargo check -p capsule-core— pass (exit 0).cargo nextest run -p capsule-core -E 'test(catalog_newer_than_this_build) | test(too_new_catalog) | test(test_open_)'— 7 passed, 0 failed.The remaining slices, the aggregate gates (
mise run check-rust,mise run test-rust,mise run check-docs-truth,mise run check-docs) and their classifications are appended here as they land; this PR is published at its first coherent commit and repaired in the open..github/workflows/ci.ymljobswift,build-ios.ymlTUIST_FFI=1) —unavailable locally(no Xcode on this host); it is the only compile proof for the.swiftdeletions.Risks and rollout
capsule-core-ffiloses two records and two functions from itsuniffinamespace; the shipping iOS composition root injectsJSONSidecarCoder, so no production path calls them.rebuild_indexno longer indexes unsigned assets (it reports them); a library holding unsigned sidecars still opens and lists them; the migration is an explicit verb with no CLI driver yet (follow-up filed)..library/quarantine/; refusals write nothing; existing signed sidecars encode byte-identically (no schema bump — the fold is an additive_unknownkey).feat/media-rawshift-still-decode-410and merges after it.Related Issues
Closes #412
Refs #436 (base)
Decisions taken
Unresolved review notes
(none yet)