Skip to content

feat: record every forgetting in the forget ledger before it commits - #261

Merged
benatespina merged 8 commits into
mainfrom
feat/forget-ledger-first
Nov 10, 2026
Merged

benatespina merged 8 commits into
mainfrom
feat/forget-ledger-first

Conversation

@benatespina

Copy link
Copy Markdown
Member

With a forget ledger configured, every forgetting is now in the ledger before it commits, so a process that stops between the two, or a transaction that fails, never loses one.

What changes

  • Intention first, everywhere. forget, forgetAgain / bounda forget, a policy's or process's forget (a new beforeCommit step of the unit of work, run once the attempt has decided, just before its transaction opens) and the retention write the forgetting's intention to the ledger first. The event's id and time are set before the transaction, which appends the event with them: the ledger and the database name one forgetting by one id.
  • An intention whose commit never happens is committed by the next reconciliation, under its own id, time and reason. Reapplying is never an error, after a restore too. A forget that rejects after its intention was written still forgets the person then; calling it again is safe.
  • Keys named by fingerprint. Ledger entries and PersonalDataForgotten carry the fingerprints of the keys a forgetting destroys. A reconciliation commits an entry only while the person still holds a key it names: a key given after (they came back) is kept. This also settles the order between several intentions of one person, whatever order they are read in.
  • Retention. A forgetting the retention made is committed again only while the retention would still forget the person (retentionDueOf), so an intention its transaction backed out of (restricted, active again) never forgets.
  • Grace. A reconciliation leaves entries younger than FORGET_LEDGER_GRACE_MS (one minute) to a later one, due once they come of age, so it never races a forgetting still committing.
  • strict now also means nothing is forgotten that the ledger has not recorded: ForgetLedgerUnavailableError (FORGET_LEDGER_UNAVAILABLE), nothing forgotten, retry is safe. Policies fail and retry; the retention holds the person.
  • forgetLedger.timeout (DEFAULT_FORGET_LEDGER_TIMEOUT_MS, ten seconds) bounds every ledger call, the outbox and prune included, with ForgetLedgerTimeoutError (FORGET_LEDGER_TIMEOUT, with operation and timeoutMs), and how long a non-strict start waits. A strict start no longer waits forever on a hung read.
  • The reason "restored" is gone (never released): a forgetting committed again keeps its own. The after-commit append (withLedgerAppend) is replaced by the intention; the stream outbox stays for what a non-strict ledger did not take.
  • Stores. The SQL ledger table gains reason and fingerprints in version 2 of the forget-ledger scope (SQLite, PostgreSQL); KV keeps them in each key's metadata; memory keeps them. Earlier entries read as a request naming no keys.
  • CLI. bounda forgotten reads the event store without starting the app (no ports, read models or ledger), so a strict ledger that is down does not stop it; its lines carry reason and fingerprints. bounda forget --from commits each whole line again under its own id, time and reason. New bounda forget-ledger expire --binding <name> --keep-for <duration> gives every key of a Workers KV ledger the expiration keepFor sets, those written before it was set included, through the project's wrangler (--local, --env, --wrangler-config, --persist-to).
  • Cloudflare. planKvForgetLedgerExpiry and expireKvForgetLedger, also from a new Node-safe subpath @bounda-dev/cloudflare/forget-ledger. parseDuration is exported from core, and isAdapter from @bounda-dev/core/adapter (what the CLI opens a storage with).

Tests for each edge

  • Crash between intention and commit: the next start commits it under the intention's id (memory, SQLite file, Durable Object over KV).
  • Ledger down without strict (commits, warns, the outbox copies it) and with strict (FORGET_LEDGER_UNAVAILABLE, nothing forgotten, retry works), for forget, forgetAgain, the retention and the Durable Object.
  • Timeout: a strict forget gives up with FORGET_LEDGER_TIMEOUT as cause; a non-strict one commits and stops waiting while the call hangs; a strict start fails past the timeout; a non-strict start goes on after it.
  • Intention reapplied: same id, time, reason and fingerprints; only within the key it names; retention only while due; grace and the due time it sets.
  • Restore with pending intentions: a SQLite backup restored after one committed forgetting and one whose commit never happened commits both, each under its id.
  • The ledger contract pins reason and fingerprints; the storage schema contract pins forget-ledger version 2; wrangler kv against its local state for forget-ledger expire.

Checked

  • pnpm check with Docker green (3422 tests) but for two local-only failures: astro check crashes on macOS with Node 25 (below), and the Vite plugin test of react-router, untouched, timed out once under a machine load of 170 and passes alone.
  • Stryker on the changed code: core 97 %, CLI 94 %, Cloudflare 95 %, PostgreSQL 100 %; the survivors left are equivalent (guards the types need, reads that only save work) or in lines this branch does not change.
  • /code-review high: 7 findings, 5 fixed (retention checks the row before writing an intention, no third read when committing an entry again, isAdapter reused, own-ledger docs, no JSDoc on CLI internals), 2 not reachable (the Worker serves no forget; key reads and removal are serialized in the transaction).

Left out

  • KV metadata holds 1024 bytes, which caps the fingerprints of one entry at about 35 keys: only relevant once categories give a person several keys (follow-up for categories).
  • astro check of the docs crashes locally on macOS with Node 25 in @emnapi/runtime 2.0.0-beta.2, on main too; CI runs it on Linux.

A forgetting committed and then appended to the ledger was lost to a restore when the process
stopped between the two steps. Every forgetting now writes its intention first, with its event's
id and time set before the transaction, and a reconciliation commits again, under the same id,
time and reason, each one the database does not hold while the person still holds a key it names
by fingerprint. A strict ledger forgets nobody it has not recorded, every ledger call is bounded
by forgetLedger.timeout, bounda forgotten reads the event store without starting the app, and
bounda forget-ledger expire gives KV entries written before keepFor their expiration.
…ation testing

Pass the ledger to the intentions with its strictness and open it only when configured, so
nothing guards a ledger that cannot be missing. A whole entry is committed again without reading
its event and key a third time, a bare id given a key meanwhile takes a new intention, and the
retention checks the person's row is current before it writes an intention it would leave. The
CLI opens a project's storage through isAdapter, now public, and a ledger of your own is told to
keep every field of an entry.
Closing the client under a statement or transaction still running frees what libSQL's native
code is using; a ledger call given up for its timeout and then closed with the app did just that.
The connection now waits for everything it started before it closes.

The two tests that opened a file from six worker threads and terminated them crashed the test
process on Windows now and then (0xC0000005): libSQL's native code does not survive a thread that
ends under it. They open the file from processes instead, as instances do.
…ledger-first

# Conflicts:
#	docs/src/content/docs/guides/personal-data.md
#	docs/src/content/docs/reference/configuration.md
#	packages/cli/src/run.test.ts
#	packages/cli/src/run.ts
#	packages/cloudflare/src/forget-ledger.ts
#	packages/core/src/adapter/sql/forget-ledger.ts
#	packages/core/src/adapter/sql/index.ts
#	packages/core/src/adapter/sqlite/migrations.ts
#	packages/core/src/index.ts
#	packages/core/src/kernel/personal-data/ledger-append.test.ts
#	packages/core/src/kernel/personal-data/reconciliation-app.test.ts
#	packages/core/src/kernel/personal-data/retention.test.ts
#	packages/core/src/kernel/personal-data/retention.ts
#	packages/postgresql/src/migrations.ts
…ledger-first

# Conflicts:
#	packages/cloudflare/src/index.ts
@benatespina
benatespina merged commit a601eef into main Nov 10, 2026
11 checks passed
@benatespina
benatespina deleted the feat/forget-ledger-first branch November 10, 2026 00:17
benatespina added a commit that referenced this pull request Nov 10, 2026
Integrates B4 (#260), A2b (#259) and B2 (#261): the forget ledger's intention
carries the categories a forgetting asks for and the reconciliation commits
again only those; the categories migrations follow B4's and B2's in each scope;
bounda status reports each retention.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant