Skip to content

fix(graph-merge): upgrade a pre-marker provenance sidecar on a database with the revision journal - #785

Merged
pdlug merged 3 commits into
mainfrom
fix/legacy-sidecar-revision-journal
Oct 6, 2026
Merged

pdlug merged 3 commits into
mainfrom
fix/legacy-sidecar-revision-journal

Conversation

@pdlug

@pdlug pdlug commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

What was wrong

openProvenanceStore refused to upgrade a pre-marker provenance sidecar (a sidecar graph id holding Provenance rows and the pre-marker schema, but no ownership marker) whenever the revision-change journal was installed on the database. The refusal was ConfigurationError GRAPH_MERGE_PROVENANCE_ID_COLLISION with reason: "application-graph", whose remediation ("rename the colliding application graph") is wrong advice for a sidecar this library wrote itself.

The journal's triggers are installed per database, not per graph. They record every node write, so each Provenance row the pre-marker sidecar wrote left an entry (entity: "node", kind: "Provenance", id: <row id>) under the sidecar's own graph id. The occupancy probe treats every graph-scoped content relation other than the node table as foreign occupancy in both of its scopes, on the reasoning that a sidecar writes to none of them. That reasoning holds for every such relation except the journal, which the sidecar never writes but the database writes on its behalf.

What changed

In the probe's foreign scope (the scope used only to classify a pre-marker sidecar's contents) the journal is now narrowed the same way the node table already is. A journal entry is accounted for only when it records a node whose row is still stored under the same graph id and that (kind, id). The predicate does not name the Provenance kind itself: the probe reaches the journal only after its node-table lookup found no stored row of another kind, so a stored row an entry can match is a Provenance row, and that row is then verified by isOwnedProvenanceRow like every other row of the sidecar. An excused entry therefore says nothing the node table has not already said and been checked on.

Every other journal entry under the graph id still refuses: an entry for an edge, an identity assertion or a node of another kind records content a sidecar never holds, and an entry for a Provenance id with no row stored under the sidecar graph id records a hard-deleted row that nothing can verify, even when another graph id (a copied sidecar, a colliding application row) stores a Provenance row under the same id. A pre-marker sidecar whose Provenance row was deleted by hand on a journaled database therefore still refuses with the application-graph wording; that is deliberate, since the pre-marker release only ever upserted Provenance rows and never deleted one. The any scope, which decides whether an unregistered id is free to claim, is unchanged: any journal entry at all still refuses there.

The rule is one SQL predicate with one owner (unaccountedJournalEntryFilter), applied by the same probe on both the unfenced preflight and the fenced claim. No relation role, trigger or schema changed. The entity value the journal records per relation is now an exported constant (REVISION_JOURNAL_ENTITY in src/backend/revision-journal.ts) that both trigger builders pass as each trigger's target and the filter matches on; the generated trigger DDL is byte-identical. It is not yet the only spelling: the PostgreSQL trigger function body still compares against an inline 'identity' and the lineage reader in src/store/recorded-capture/lineage.ts still spells entity IN ('node', 'edge'). Moving those two onto the constant is a follow-up. The application-graph row in the error reference states the journal exception.

Cost

A sidecar whose journal entries are all accounted for no longer short-circuits on the first journal row: the probe visits every journal entry for that graph id and checks each against the node table (a per-entry key lookup on SQLite; on PostgreSQL a hashed or per-entry subplan at the planner's choice). It runs on the unfenced preflight and inside the claim's fence, on the one-time pre-marker upgrade only; an owned sidecar is recognized by its marker and never reaches the contents probe.

Revert check

The reproduction (upgrades a legacy sidecar whose own writes the revision triggers journaled) was written first and failed on SQLite, PGlite and server PostgreSQL with the application-graph collision before the fix; it also asserts the journal holds exactly the one (node, Provenance, <row id>) entry, so it cannot pass on a database where the triggers did not fire.

The refusal cases were each checked against a mutation of the predicate, since they pass trivially on unfixed code. Each mutation below was run on SQLite, PGlite and server PostgreSQL, and each named case went red on all three while the rest stayed green. Skipping the journal in the foreign scope turns all five refusal cases red. Dropping the stored-row EXISTS turns the hard-deleted, stored-under-another-graph and node-of-another-kind cases red. Deleting the kind correlation inside the EXISTS (nodes.kind = journal.kind, the predicate's only kind clause) turns only the "node of another kind under the stored row's id" case red. Deleting the entity test turns only the "edge under the stored row's kind and id" case red. Deleting the graph_id correlation inside the EXISTS turns only the "Provenance row stored only under another graph id" case red (the sidecar is upgraded instead of refused), which is why that case exists: the hard-deleted case uses an id stored nowhere and cannot tell a same-graph lookup from a database-wide one.

pdlug added 3 commits October 5, 2026 15:38
…se with the revision journal

The revision-change journal's triggers are installed per database and record every node write, so a pre-marker sidecar's own Provenance rows left journal entries under the sidecar graph id. The occupancy probe read those entries as an application's content and refused the upgrade with GRAPH_MERGE_PROVENANCE_ID_COLLISION (application-graph).

In the probe's foreign scope a journal entry is now accounted for only when it records a Provenance node whose row is still stored under that (kind, id); that row is verified like every other. Any other journal entry under the id still refuses, and the any scope that decides whether an unregistered id is free is unchanged.
… the journal entity one owner

A journal entry for a Provenance id is excused only when the row is stored under the same graph id. That correlation was not pinned: the hard-deleted refusal case used an id stored nowhere, so a database-wide lookup passed it. A new case stores the Provenance row under another graph id only and must still refuse.

The entity value the journal records per relation moves to REVISION_JOURNAL_ENTITY, written by both trigger builders and matched by the filter; the generated trigger DDL is unchanged. The error reference states the journal exception on the application-graph reason, and the shared test comment now describes every case.
The filter tested the journal entry's kind against Provenance and also correlated the stored row on the entry's kind. Either clause alone was sufficient, so neither could be shown load-bearing by deleting it. The constant test is removed: an entry is excused when its node row is still stored under the same graph id, kind and id, and the node probe that runs first has already refused any stored row of another kind. Deleting the remaining kind correlation now turns the node-of-another-kind refusal case red on every backend.

REVISION_JOURNAL_ENTITY's comment names what uses it and the two inline spellings that remain.
@pdlug
pdlug merged commit cb878a1 into main Oct 6, 2026
23 checks passed
@pdlug
pdlug deleted the fix/legacy-sidecar-revision-journal branch October 6, 2026 18:42
@github-actions github-actions Bot mentioned this pull request Oct 6, 2026
pdlug pushed a commit that referenced this pull request Oct 21, 2026
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and the packages will
be published to npm automatically. If you're not ready to do a release
yet, that's fine, whenever you add more changesets to main, this PR will
be updated.


# Releases
## @nicia-ai/typegraph@0.73.2

### Patch Changes

- [#785](#785)
[`cb878a1`](cb878a1)
Thanks [@pdlug](https://github.com/pdlug)! - Upgrade a pre-marker
merge-provenance sidecar on a database with the revision-change journal
installed. The journal's triggers record every node write, so the
sidecar's own `Provenance` rows left entries under its graph id and the
upgrade refused them with `GRAPH_MERGE_PROVENANCE_ID_COLLISION`
(`reason: "application-graph"`). A journal entry is now accepted when it
records a `Provenance` node whose row is still stored and verifies; any
other journal entry under that graph id still refuses.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant