progress: copy operation and engine-measured work source (v4)

SAFETY.md

@@ -33,7 +33,7 @@ The invariant registry (invariant IDs referenced below) lives in
 | `pkg/diffplan` — desired schema → routed convergence plan, the declarative front door as a library (the CLI `diff` and embedding orchestrators share it) | ❌ periphery | exists | — |
 | `pkg/migrate` — one gated statement → resolve, classify, route, execute → one verdict; the imperative front door as a library (the CLI `migrate` and embedding orchestrators share it), plus the desired-state execution loop (`RunDesired`: derive the convergence plan, admit it as a whole, run each planned statement back through the same pipeline) | ❌ periphery² | exists | — |
 | `internal/cli` — CLI, flags, help, prompts | ❌ periphery | `migrate`, `pull`, `diff`, `fmt`, `lint`, `suggest`, `capabilities`, and `status` exist | — |
-| `pkg/progress` — strategy-wide progress snapshots; the executors' observation seam (core imports it, so its locking discipline is core-critical); copy counters reserved for later | ✅ core | native progress exists | — |
+| `pkg/progress` — strategy-wide progress snapshots; the executors' observation seam (core imports it, so its locking discipline is core-critical); the `WorkSource` seam for engine-measured steps such as the copy | ✅ core | native progress and the work-source seam exist | — |
 | orchestrator adapter | ❌ periphery | planned (Phase 11) | OC-* hold *at* the boundary |
 | `internal/testutil` | ❌ test-only | exists | — |
 
@@ -93,9 +93,13 @@ The short version — the full rules live in [docs/tcb-model.md](docs/tcb-model.
   `pkg/progress` (the executors' progress-observation seam: they write state into a
   caller-owned tracker whose mutators take only a memory lock, and its polling reads ride
   the reserved verdict session behind a separate poll lock — the executor's own state
-  updates never wait for a database read, but the verdict handoff *is* observer-gated:
-  `StopConcurrentBuild` deliberately drains an in-flight poll before the executor reclaims
-  the session, a wait bounded by the poller's context and the session's `statement_timeout`),
+  updates never wait for a database read, but the handoffs that end a poll target's
+  ownership *are* observer-gated: `StopConcurrentBuild` and `SetWorkSource` drain an
+  in-flight poll before the executor reclaims the build's session, a wait bounded by the
+  poller's context and the session's `statement_timeout`, and `StopWorkSource` and
+  `SetConcurrentBuild` drain one before an engine step releases the state its `WorkSource`
+  reads, a wait the `WorkSource` contract requires `Work` to bound itself — memory reads or
+  catalog reads under a session `statement_timeout`, never the observer's context alone),
   stdlib. Adding one requires a recorded decision (see the rubric in
   [docs/tcb-model.md](docs/tcb-model.md) — copy small things, take pinned dependencies only
   for load-bearing expertise).

docs/README.md

@@ -49,7 +49,7 @@ Aurora-only. Why that combination is the product is [vision.md](vision.md); star
 | [limitations.md](limitations.md) | The **current limitations** — schema changes pg-sprite refuses today, why they are unsafe or unsupported, and where an operator must act outside the engine. |
 | [lint-report.md](lint-report.md) | The **lint report contract** — the versioned JSON shape `pg-sprite lint` emits for offline CI gating: finding fields (verbatim SQL, line/column), the codes table, severities and exit behavior, the offline-conservatism rules, and how the contract versions relative to the plan report. |
 | [suggest-report.md](suggest-report.md) | The **suggest report contract** — the versioned JSON shape `pg-sprite suggest` emits for offline advice: the typed caveat vocabulary (what changes about how you must run a safer form, and what a failed step leaves behind), the typed guidance codes for rewrites the planner cannot construct, and the operation → safer form → caveats table (pinned by test). |
-| [progress-report.md](progress-report.md) | The **progress report contract** — the versioned JSON snapshot a caller receives when polling a running change through the `*WithProgress` entry points: phases and operations vocabularies, the terminal-freeze rule, server-observed work counters, and polling semantics (pinned by test). |
+| [progress-report.md](progress-report.md) | The **progress report contract** — the versioned JSON snapshot a caller receives when polling a running change through the `*WithProgress` entry points: phases and operations vocabularies, the terminal-freeze rule, server-observed and engine-measured work counters, and polling semantics (pinned by test). |
 | [engine-role.md](engine-role.md) | The **engine-role provisioning contract** — the tiered minimum access a PostgreSQL user needs to run schema changes against tables it does not own: role membership for owner-gated DDL, schema `CREATE` for index builds and shadow objects, `SET ROLE` for owner-correct shadow creation, replication access for CDC, and the explicit list of powers the engine role must *not* have. Preflight refusals name the missing `GRANT` and point here. |
 | [invalid-index-recovery.md](invalid-index-recovery.md) | The **operator runbook** for the native path's invalid-index outcomes — an invalid index the executor found or left. How the typed states are told apart, which ones `RebuildAbandonedIndex` recovers on its own (and the lock-and-identity proof that makes its drop safe), when the entry may be another actor's healthy in-flight build, and what to check when the executor could prove nothing. |
 | [testing.md](testing.md) | The **test-suite guide** — how to run the suite (unit, per-major, all supported majors, compose database), current coverage, the remaining executor-phase test obligations, and the vanilla-PostgreSQL-matrix vs real-Aurora validation boundary. |

docs/progress-report.md

@@ -4,8 +4,9 @@ The progress snapshot is the machine-readable observation a caller receives when
 running schema change through the `*WithProgress` executor entry points. It is the one JSON
 shape an operator or orchestrator consumes to display or act on execution progress. This
 document is the contract: the fields, the closed vocabularies, and the behavior required of
-a consumer. The Go source of truth is `pkg/progress`; `TestSnapshotJSONShape` pins the exact
-keys, including the example at the end of this page.
+a consumer. The Go source of truth is `pkg/progress`; `TestSnapshotJSONShape` and
+`TestSnapshotJSONShapeForACopyStep` pin the exact keys, including the two examples at the end
+of this page.
 
 ## Versioning: `format_version`
 
@@ -16,8 +17,11 @@ phase or operation value is a contract change and bumps `format_version`, even i
 is added or renamed.
 
 Adding a field bumps `format_version` so a strict consumer can detect the new shape from the
-version. The current version is **3**: version 2 added `detail.statement`; version 3 added
-`detail.current_locker_pid`, `work.lockers_total`, and `work.lockers_done`.
+version. The current version is **4**: version 2 added `detail.statement`; version 3 added
+`detail.current_locker_pid`, `work.lockers_total`, and `work.lockers_done`; version 4 added
+the `copy` operation and split `work` into two counter families — the server-observed build
+counters and the engine-measured copy counters (`rows_*`, `bytes_*`) — selected by
+`detail.operation`, so `work` is no longer a signal that a concurrent index build is running.
 
 The [plan report](plan-report.md), [lint report](lint-report.md), and
 [suggest report](suggest-report.md) are separate contracts with their own `format_version`;
@@ -52,18 +56,35 @@ licenses a consumer to intervene in the change itself.
 | `active` | bool | always | Whether an operation is executing now. `false` with `phase: "running"` means a concurrent build's progress row has left the server view. |
 | `attempt` | int | bounded retries only | The current attempt number when the executor is inside its bounded retry loop. |
 | `current_locker_pid` | int | while waiting on a locker | PostgreSQL backend PID currently blocking the concurrent build; omitted when none is published. |
-| `work` | object | server-observed work only | Present exactly when the server published a progress row; then **every** counter below is present, so a fresh build reports honest zeros rather than an empty object. |
+| `work` | object | measured work only | Present exactly when something measured the step's work — the server published a progress row for a concurrent build, or the engine reported the copy step's counters; then **every** counter below is present, so a fresh build or an empty copy reports honest zeros rather than an empty object. Which counters mean anything is decided by `operation`, not by `work` being present — see [Work counters](#work-counters). |
 
-`statement` is the submitter's statement after qualification and canonicalization, so a
-consumer rendering it into a shared surface must clamp and escape it.
+`statement` is the SQL the engine is running for the step: for a native operation the
+submitter's statement after qualification and canonicalization, for a `copy` step the
+engine's own frozen chunk insert (the template with its `$1`/`$2` key bounds, not a chunk's
+rendered values). Either way it is real SQL that reached the server, so a consumer rendering
+it into a shared surface must clamp and escape it.
 
 ### Work counters
 
-`blocks_done` / `blocks_total`, `tuples_done` / `tuples_total`, and
-`lockers_done` / `lockers_total` come from
-`pg_stat_progress_create_index` during a concurrent index build. `rows_copied` /
-`rows_total` and `bytes_copied` / `bytes_total` are reserved for copy-and-swap and are `0`
-on every native operation — the engine never fabricates copy counters.
+Two operations publish `work`, and each measures only its own counters; the other
+operation's counters are `0`, never estimated. A consumer selects the counter family from
+`detail.operation`, never from the presence of `work`: a present `work` says only that
+something measured the step, and a consumer that reads its presence as "a concurrent index
+build is running" will show a `copy` step as a build with zero blocks. Render the build
+counters for `concurrent-index-build`, the copy counters for `copy`, and nothing from `work`
+for an operation you do not recognize.
+
+**Server-observed** (`concurrent-index-build`): `blocks_done` / `blocks_total`,
+`tuples_done` / `tuples_total`, and `lockers_done` / `lockers_total` come from
+`pg_stat_progress_create_index`, read over the executor's reserved session.
+
+**Engine-measured** (`copy`): `rows_copied` / `rows_total` and `bytes_copied` /
+`bytes_total` come from the copy-and-swap row copy itself, which knows the rows it has landed
+from the chunks it committed. The copy step is not a single server statement, so PostgreSQL
+publishes no progress view for it; the tracker instead polls the engine's work source for
+the step. Every counter is something the engine measured — a row count from committed chunks,
+a size read from the catalog — never a projection; a counter the engine cannot measure stays
+`0`. Native operations report no rows or bytes.
 
 ## Phases
 
@@ -85,15 +106,28 @@ returns the identical snapshot, elapsed values included.
 | `optimistic` | One bounded direct native attempt. |
 | `brief` | A brief transactional sequence step. |
 | `validate-constraint` | A constraint-validation scan. |
-| `concurrent-index-build` | A concurrent index build (the one operation with server-observed `work`). |
+| `concurrent-index-build` | A concurrent index build (`work` is server-observed). |
+| `copy` | The copy-and-swap row copy from the source table into its shadow (`work` is engine-measured). |
 
 ## Polling semantics
 
 The tracker is caller-owned and has no goroutines or timers: polling lifetime is exactly the
 caller's context. A poll during an active concurrent index build performs one read of the
-server's progress view over the executor's reserved session; every other poll is pure
-memory. On a query error the returned snapshot still carries the last-known tracker state —
-`phase` is never empty — with the error returned alongside for the caller to classify.
+server's progress view over the executor's reserved session; a poll during a `copy` step
+asks the engine's work source once; every other poll is pure memory. On a query or source
+error the returned snapshot still carries the last-known tracker state — `phase` is never
+empty — with the error returned alongside for the caller to classify. Pollers serialize
+against each other, so the reserved session and the work source each see one observation
+at a time.
+
+The handoffs that end a poll target's ownership of the step — stopping a build or a work
+source, or replacing one with the other — wait for an observation in flight before they
+return, so the engine never releases a session or the state a source reads while a poll is
+still using it. That makes the work source part of the engine's stop path: its `Work` must
+bound itself (memory reads, or catalog reads on a session with `statement_timeout` set) and
+honour the poll's context, because a poll that returns only when its observer gives up
+would stall the engine behind an observer that never does. A `Work` that reads the catalog
+is a CO-9 read site — `pg_catalog`-qualified, tested under a shadowing `search_path`.
 
 The tracker is also the operator's stop path for a running concurrent index build:
 `Tracker.CancelBuild` signals the build's backend over the same reserved session, and only
@@ -122,7 +156,7 @@ A poll during step 2 of a 3-step sequence, mid concurrent index build:
 
 ```json
 {
-  "format_version": 3,
+  "format_version": 4,
   "phase": "running",
   "step": 2,
   "total_steps": 3,
@@ -150,3 +184,34 @@ A poll during step 2 of a 3-step sequence, mid concurrent index build:
   }
 }
 ```
+
+A poll during step 2 of a 4-step copy-and-swap, mid row copy (`TestSnapshotJSONShapeForACopyStep`
+pins it):
+
+```json
+{
+  "format_version": 4,
+  "phase": "running",
+  "step": 2,
+  "total_steps": 4,
+  "elapsed_ns": 2750000000,
+  "step_elapsed_ns": 750000000,
+  "detail": {
+    "operation": "copy",
+    "statement": "INSERT INTO public.t_shadow (id) SELECT id FROM public.t WHERE id BETWEEN $1::bigint AND $2::bigint ON CONFLICT (id) DO NOTHING",
+    "active": true,
+    "work": {
+      "rows_copied": 1200,
+      "rows_total": 5000,
+      "bytes_copied": 98304,
+      "bytes_total": 409600,
+      "blocks_done": 0,
+      "blocks_total": 0,
+      "tuples_done": 0,
+      "tuples_total": 0,
+      "lockers_total": 0,
+      "lockers_done": 0
+    }
+  }
+}
+```