Conversation
…(CO-1) Verifier digests every chunk up to the copier's landed watermark on both sides inside one read-only REPEATABLE READ transaction, casting every column to the shadow's type (D7), under the copier's guard: owner role, catalog-only search_path, ACCESS SHARE on both relations before the snapshot, lock confirmation, relation-OID check. It reports the chunks that differ; policy, repair, and the proof constructors follow.
Resolves the pkg/checksum rows in SAFETY.md, docs/architecture.md, and docs/copy-and-swap-design.md against the copier progress-filler rows from #132: this branch's checksum text, main's copier and progress text.
…nt the proofs (CO-2, CO-3) Check runs one pass under a DivergencePolicy the caller states every time; the zero value is refused. A clean pass mints a CleanWatermark, and a VerifiedShadow when the watermark is complete. Under abort a difference is a DivergenceError with the shadow untouched. Under repair every differing chunk is replaced in one guarded transaction — delete the shadow's rows over the chunk, then the copier's own chunk statement — and read again in a fresh snapshot; a chunk that still differs is a RepairError, and a pass with repairs mints no proof. copier exports InsertChunk so a repair recopies with the statement the copy used, and Watermark.Complete names the top-of-key-space test.
…toverReady (CO-1, ST-5) GateCutover takes a BuiltShadow and a checksum.VerifiedShadow for the same table and, in one read-only transaction under the per-table lock, re-reads both relations against what the build recorded. It refuses, fail-closed, when either proof is empty or they name different relations, when the verified watermark stops short of the key space, when either OID moved, when either table's fingerprint or metadata snapshot drifted, when a shadow index is invalid, or when a name the swap must assign is already taken. The checklist is one function so the swap can run it again inside its own transaction. CutoverReady is what the gate mints: the two proofs it held, every source index and extended statistics object paired with its shadow counterpart by catalog definition rather than by name (LIKE renames them), and the sequences the source's columns own, which the swap re-owns. The builder now records the shadow's own metadata snapshot alongside the source's, since the gated statement may change metadata on purpose, and carries each column's explicit statistics target onto the shadow, which LIKE … INCLUDING ALL does not copy.
ALTER COLUMN ... TYPE re-creates every index on that column for the new type, so pg_index.indclass on the shadow's copy flips to the new type's default operator class (int4_ops -> int8_ops) while the index still covers the same columns the same way. The gate's exact-definition pairing left such indexes unpaired, and a swap would have kept the LIKE-derived name (_pgsprite_<hash>_new_pkey) on the live table. GateCutover now derives the set of columns the statement retyped (same name on both sides, different canonical type) and runs a second pairing pass over the exact pass's leftovers only, with the operator class and collation of every retyped key column set aside. Access method, uniqueness, columns, ordering, predicate, and backing constraint must still agree; an expression over a retyped column and every untouched column stay exact. Pairs keep the source order. Unit tests cover the retyped-column derivation (including a quoted name), the relaxed pass over leftovers, and that only opclass/collation are relaxed. Integration tests gate an int -> bigint change on a table with a PK, a DESC index on the retyped column, and an index on an untouched column, asserting the int4_ops/int8_ops precondition and full pairing; and that a dropped column's index still stays unpaired alongside a retyped one. Design D8 and invariant ST-5 record the one set-aside difference.
…8, D9, LK-2, LK-4) Cutover is the one ACCESS EXCLUSIVE window of the copy-and-swap route. It takes a CutoverReady from the gate and, in one transaction under the table lock session, locks the source and the shadow under lock_timeout, runs the caller's DrainFunc once writers are excluded, re-runs the gate's checklist so the pairing it renames by is as fresh as the lock, renames every source dependent to its derived _old name, the source to _old, the shadow to the source's name, and each paired shadow dependent to its partner's name (D8), then hands the sequences over (D5): shared serial/nextval sequences are re-owned to the live column, and each identity column is recreated with the source sequence's declared options under the source sequence's name and set to its (last_value, is_called) through setval. The catalog is re-read before COMMIT and the swap refuses (cutover-swap-mismatch) unless it shows exactly the state it set out to produce. The result is the SwappedTable proof; DropOldTable accepts only that proof and drops only the relation whose OID it recorded (D9). A lock timeout or a deadlock rolls back and retries after a linearly growing backoff for a bounded number of attempts, then returns ErrLockRetriesExhausted with the source still live (LK-2). A lost connection is never assumed to have rolled back: a fresh connection reads which OID bears the source name — the shadow's is reported as the committed swap it was, the source's as a rollback carrying the connection error, and anything else is refused as cutover-outcome-ambiguous (LK-4). Integration tests swap a DROP COLUMN change and assert names, OIDs, the constraint following its index, statistics, sequence ownership, a continuing serial counter, and the old-table drop; hand off an ALWAYS identity with INCREMENT BY 2 START WITH 10 with no gap plus a never- advanced BY DEFAULT identity that still issues its start first; retry behind a reader released from the injected sleep; exhaust the attempts behind a reader that never lets go; run a drain under the lock and abort on its error; resolve a terminated swap backend by inspection; and walk inspectOutcome through all three catalog states.
Kiran01bm
force-pushed
the
kiran01bm/cs9-pair-type-change
branch
from
October 1, 2026 11:10
de2c3a0 to
5d6f315
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cutover swap:
Cutoverconsumes theCutoverReadyproof the gate (#137, #138) mints and performs the oneACCESS EXCLUSIVEwindow of the copy-and-swap route;DropOldTableis the D9 drop after it.Why
Build (
BuildShadow), copy (Copier), verify (Verifier), and gate (GateCutover) are landed; nothing yet renames the shadow into place. This PR is the swap transaction described indocs/copy-and-swap-design.md§ "Cutover transaction, step by step", with the LK-2 bounded lock retry and the LK-4 lost-connection resolution the design requires.What
pkg/schemachange/cutover.go—Cutover(ctx, pool, lock, ready, drain, CutoverOptions) (SwappedTable, error).CutoverOptionsembedsOptionsand addsLockAttempts(default 5),LockBackoff(default 500ms, linear ×attempt), and an injectableSleep.DrainFuncruns inside the swap transaction after the lock; nil drains nothing. The retry loop retries only55P03/40P01; exhaustion returnsErrLockRetriesExhaustedwrapping the server's last answer.pkg/schemachange/cutover_swap.go— one attempt: bounded tx as owner → confirm table lock →LOCK TABLE source, shadow IN ACCESS EXCLUSIVE MODE→ drain → re-rungateCutoverTx(fresh pairing under the lock) → D8 renames (every source index incl. unpaired, statistics, identity sequences →OldDependentName; source →OldName; shadow → source name; paired shadow dependents → source names) → sequence handoff → confirm → commit.pkg/schemachange/cutover_handoff.go— D5:ALTER SEQUENCE … OWNED BY live.colfor shared sequences; for each identity column, read(last_value, is_called)from the renamed old sequence,DROP DEFAULT,ADD GENERATED … AS IDENTITY (<source's declared options>), rename the server-named sequence to the source sequence's name,setvalto the position read.pkg/schemachange/cutover_outcome.go—SwappedTable(private constructor;Table,OldTable,LiveOID,OldOID,Indexes,Statistics,OwnedSequences,IdentityColumns,Attempts);confirmSwapre-reads the catalog beforeCOMMITand refusescutover-swap-mismatchunless live name = shadow OID,_oldname = source OID, every paired name is on the live table, and owned sequences / identity columns equal the proof;inspectOutcomeis the LK-4 read from a fresh connection.pkg/schemachange/drop_old.go—DropOldTable(ctx, pool, lock, swapped, Options): separate bounded tx, drops only the relation whose OID the proof recorded (cutover-relation-replacedotherwise), noCASCADE.refusal.go/docs/refusal-classes.md—cutover-swap-mismatch(ST-6,invariant-violation) andcutover-outcome-ambiguous(LK-4,environmental).copy-and-swap-design.md; LK-2 and LK-4 Enforced paragraphs ininvariants.md;pkg/schemachangerow inSAFETY.md;SequenceOptionscomment inidentity.gonow matches the handoff (START WITH replays the source's declaredseqstart; the position is set bysetval).Decisions worth a look
START WITHreplays the source's declared start, notlast_value. The recreated sequence'spg_sequencerow then equals the source's, which is whatconfirmSwapcompares and whatALTER SEQUENCE … RESTARTwould honour; the counter's position is set separately bysetval(last_value, is_called). The oldSequenceOptionscomment said the opposite and is corrected here.LOCK TABLE, so a straggling copier/applier connection cannot hold the rename.GateCutoverresult; the gate refuses any drift between the two, so when both pass they agree, and the rename uses the in-lock pairing.57P01,57P02, class08),net.Error,io.EOF/io.ErrUnexpectedEOF, orpgconn.SafeToRetry; a lock timeout or statement error the server answered is known to have rolled back and skips the catalog read.Tests
cutover_test.go):CutoverOptions.validatedefaults and negative-bound refusals;lockRetryablematches only55P03/40P01;connectionLostseparates lost links from server answers;identityClauserenders every declared option (ALWAYS and BY DEFAULT, negative increment, CYCLE);sleepContexthonours cancellation.cutover_integration_test.go, PG16 testcontainers):TestCutoverSwapsTheShadowInAndDropOldTableRemovesTheSource—orders DROP COLUMN note: live OID is the shadow,_oldOID is the source, live indexes are exactlyorders_pkey/orders_qty_idxand the PK constraint moved with its index, the unpairedorders_note_idxwent to a derived name on the old table, statistics renamed both ways,orders_id_seqowned by the live column and issuing 2501 next; afterDropOldTablethe old table and its indexes are gone, the sequence survives, the next id is 2502.TestCutoverHandsOffIdentityColumnsWithoutAGap—GENERATED ALWAYS AS IDENTITY (INCREMENT BY 2 START WITH 10)plus aBY DEFAULTidentity the source never advanced: both are identity columns again under the source sequence names withseqstart=10,seqincrement=2; the next id is 5010 (no gap) and the never-advanced one issues 1; the old identity sequences go with the old table.TestCutoverRetriesTheLockBehindAReaderAndSucceedsOnceItReleases— anACCESS SHAREholder released from inside the injectedSleep;Attempts()==2, one backoff of one step.TestCutoverStopsAfterTheBoundedLockAttempts— holder never releases;ErrLockRetriesExhaustedwrapping55P03, backoffs[1ms, 2ms]for three attempts, source still live, shadow kept.TestCutoverRunsTheDrainUnderTheLockAndAbortsOnItsError— the drain observes its own grantedAccessExclusiveLockon the source viapg_locksand its row is in the live table; a drain error aborts with nothing swapped.TestCutoverResolvesALostConnectionByInspectingTheCatalog— the drain hook terminates the swap's own backend; the swap reports57P01as a rollback (not an invariant violation, not exhaustion), source still live, no rename survived.TestCutoverRefusesAZeroProofAndAMissingLock,TestDropOldTableRefusesARelationThatIsNotTheRetainedSource.cutover_outcome_integration_test.go):inspectOutcomewalked through not-swapped → swapped → name absent → name taken by an unproven relation.SKIP_INTEGRATION=1 go test ./...,go test -race ./pkg/schemachange/(124s),make lint(0 issues) pass locally; the three timing-sensitive tests pass 10/10 with-count=10.Stack
Based on #138 (
kiran01bm/cs9-pair-type-change) → #137 (kiran01bm/cs9-fidelity-gate) →main. Retarget as each parent merges. No capability, verdict, or CLI surface changes until the router flip;demo/tour.shis unaffected. The orchestrator that chains build → copy → verify → gate → cutover → drop, and the WAL-drainDrainFunc, are the next rows.