feat: the embedding API facade — Connection, Statement, Rows, Transaction (spec 013, all 8 requirements)

.openspec/adr/0042-rows-changed-counted-by-codegen-flag.md

@@ -0,0 +1,103 @@
+# 0042 — Codegen decides which mutation is a row change, and `None` is not zero
+
+**Status:** Accepted · **Date:** 2026-09-04
+
+## Context
+
+Spec 013 Requirement 1 asks for `sqlite3_changes()`: how many rows the last
+`INSERT`/`UPDATE`/`DELETE` changed. Spec 013 calls it the one item on its list
+a consumer cannot work around, because without it a caller cannot distinguish
+an `UPDATE` that matched from one that did not, and that distinction is what
+every optimistic-concurrency scheme is built on.
+
+The obvious implementation — increment a counter in the `Insert` and `Delete`
+opcode handlers — is wrong, and measurably so on the tree at 0.18.10:
+
+| statement | opcodes emitted per row | counted naively |
+|---|---|---|
+| `INSERT` | `Insert` | 1 |
+| `DELETE` | `Delete` | 1 |
+| `UPDATE`, single-pass | `Delete` + `Insert` (`update.rs:603,605`) | **2** |
+| `UPDATE`, two-pass range-seek | ephemeral `Insert` (`update.rs:276`) + `Delete` + `Insert` | **3** |
+
+The two-pass plan is #666/#675's range-seek path, which stashes matched rowids
+in an ephemeral b-tree using the same `Opcode::Insert`. So the same `UPDATE`
+would report 2 or 3 depending on which plan the optimizer picked, and neither
+is 1. Index maintenance (`IdxInsert`, `IdxDelete`, `AutoIndexInsert`) has the
+same character: a write, adjacent to a row, that is not a row change.
+
+The opcode does not carry enough information to answer. Codegen does.
+
+## Decision
+
+**Codegen marks the one mutation that is the row change, with
+`OPFLAG_NCHANGE` (`0x01`) on `P5`.** Same bit and same job as stock SQLite's
+flag of that name. `cursor::insert`/`cursor::delete` increment `Vm`'s counter
+only when it is set; `Instruction::with_p5` is the constructor that sets it,
+alongside the existing `with_p4`. `P5` was unread by both opcodes, so nothing
+had to move.
+
+An `UPDATE` flags its `Insert` and not the paired `Delete` — one changed row,
+counted once. `INSERT` flags its table `Insert`; `DELETE` flags both of its
+`Delete` sites. Nothing else is ever flagged.
+
+**The count is exposed as `Option<u64>`, and `None` is not `Some(0)`.**
+`StepOutcome::changes` is `Some(n)` when the program is a counting statement
+and `None` when it is not:
+
+- `Some(0)` means "this was an `INSERT`/`UPDATE`/`DELETE` and it changed
+  nothing" — a lost optimistic-concurrency race, which is the case the
+  requirement exists to make visible.
+- `None` means "not that kind of statement", so a connection tracking
+  `sqlite3_changes()` leaves its stored count alone.
+
+The discriminator is **static** — `Program::counts_changes()` asks whether the
+program *contains* a flagged instruction, not whether one executed. An
+`UPDATE` whose `WHERE` matches nothing never runs its flagged `Insert` but
+must still report `Some(0)`.
+
+**`execute_transaction_step` becomes a wrapper** over
+`execute_transaction_step_counted`, which returns the count. Same pattern
+ADR-0040 settled on for streaming: one loop, the older signature expressed in
+terms of the newer one, so the two cannot drift and the existing suite is the
+equivalence proof.
+
+## Alternatives rejected
+
+- **Count in the handlers, unconditionally.** Reports 2 or 3 for a one-row
+  `UPDATE`, plan-dependently, and counts index maintenance. This is the
+  alternative the table above exists to close, and
+  `update_of_one_row_reports_one_under_both_plans` is its regression guard:
+  removing the flag check fails that test and two others.
+- **Return `u64` and let `0` mean both.** Collapses "changed nothing" into
+  "not a counting statement", which is exactly the distinction SQLite's
+  retention rule is built on — a `SELECT` would zero a count that should have
+  survived it. The two-case type is the whole point and should not be
+  simplified away.
+- **A `Program { counts_changes: bool }` field set by codegen.** Equivalent
+  in behaviour, but it can disagree with the instructions it describes, and
+  `Program::new` has many call sites. Deriving it costs one pass over a
+  handful of instructions.
+- **Change `execute_transaction_step`'s return type in place.** Ten call
+  sites across `src/bin/`, tests, benches and examples, for a value almost
+  none of them want. The wrapper is free.
+- **Track the count across statements in the `Vm`.** A `Vm` lives for one
+  statement, so it cannot. Cross-statement retention is the connection's
+  rule and belongs to spec 013/Req 1's `Connection::changes`.
+
+## Consequences
+
+The number is correct for the statement just run, verified against the pinned
+3.53.4 oracle's own `changes()` for a thirteen-statement sequence covering
+both `UPDATE` plans, a miss, a partial `DELETE` and a full one
+(`tests/corpus/changes_oracle_test.rs`). Both wrong designs above were
+mutation-checked against that test as well as the unit suite.
+
+`Connection::changes` is still absent — this is the engine half. What the
+facade has left to do is one line: store the value on `Some`, ignore `None`.
+
+Adding a `P5` flag reopens no frozen set: no new opcode, so ADR-0015, ADR-0018
+and ADR-0020 are untouched. But `P5` on `Insert` is now meaningful where its
+doc comment previously said conflict-resolution flags were "not modeled", so a
+future `OR REPLACE`/`OR IGNORE` implementation must pick bits other than
+`0x01`.

.openspec/adr/0043-embedding-api-failure-surface.md

@@ -0,0 +1,127 @@
+# 0043 — The embedding API's failure surface: a flat error, both result codes, and autocommit-only busy retry
+
+**Status:** Accepted · **Date:** 2026-09-09
+
+## Context
+
+Spec 013 Requirement 5 asks for three things that all land on the same type:
+`VfsError::Locked` must surface as "a distinct, documented busy variant", a
+busy timeout must be settable per connection, and a consumer must be able to
+tell a UNIQUE violation from any other failure. ADR-0041 settled *where* the
+API lives; it said nothing about what a failure looks like coming out of it.
+
+Four choices had to be made, and each closes an alternative that is defensible
+enough to be worth writing down.
+
+**The error has to cross a thread.** Every failure travels back from the
+connection's worker over a channel, so the error type must be `Send + Sync +
+'static` unconditionally. An error that borrowed from engine state, or held an
+`Rc`, could not be returned at all. That rules out the shape the sixteen engine
+error enums have, several of which wrap layer errors by value.
+
+**The result code is two numbers, not one.** `sqlite3_errcode()` returns the
+primary code (19 for any constraint violation) and
+`sqlite3_extended_errcode()` the extended one (2067 for UNIQUE specifically).
+A caller asking "is this a constraint problem?" wants the first; one
+distinguishing UNIQUE from NOT NULL wants the second. Picking one would make
+the other unreachable, and the question of which a future `sqlx-sqlite-rs`
+driver needs was genuinely open.
+
+**A retried statement can be applied twice.** `Pager::flush`
+(`src/pager.rs:524`) surfaces `VfsError::Locked` before any byte is journaled
+and deliberately leaves `self.dirty` intact "so the caller can retry or roll
+back". In autocommit, that dirty set is the statement's own work — already
+applied in memory. Re-running the statement without discarding it first inserts
+the row once per attempt. Measured before the rollback was added.
+
+**A prepared statement can outlive its plan.** A compiled program addresses
+tables by root page, and `DROP` returns that page to the freelist for a later
+`CREATE` to reuse. A statement prepared before a schema change and run after it
+can read a page belonging to a different table, with no error anywhere.
+
+## Decision
+
+**The error type is flat.** `api::Error`'s every payload is a `String`, an
+`i32` or a `Copy` enum; layer errors arrive as already-formatted `Display`
+text. It therefore derives `PartialEq` and is unconditionally `Send + Sync +
+'static`. `#[non_exhaustive]`, so variants can be added without a breaking
+change.
+
+**Both result codes are exposed, under the names SQLite uses.**
+`Error::sqlite_code()` returns the primary, `Error::extended_sqlite_code()` the
+extended, and the primary is derived from the extended as the low byte — the
+rule `sqlite3.h` encodes (`primary | (n<<8)`). `Error::is_retryable()` is true
+for `Busy` and nothing else.
+
+**`Busy` is classified structurally, never by message text.** The match is on
+`ExecError::FlushFailed(PagerError::Vfs(VfsError::Locked { .. }))` and the
+`DumpError` equivalents, not on a substring. Requirement 5 makes busy a
+distinct *retryable* variant, so a classification that a reworded `Display`
+could silently break is the wrong trade: every busy error would become
+permanent and no test would fail.
+
+**The busy timeout retries only in autocommit, and rolls back first.** In
+autocommit the statement is the transaction, so `Pager::rollback` followed by
+re-running it is a faithful retry of the whole unit. Inside an explicit
+transaction it is not — the statement's mutations share the pending set with
+every earlier statement's — so a busy there is reported immediately and is the
+*transaction's* to retry. Stock SQLite behaves the same way with
+`SQLITE_BUSY` at `COMMIT`. Backoff follows `sqliteDefaultBusyCallback`'s
+ladder; the default timeout is zero, as SQLite's is.
+
+**A stale prepared statement is recompiled, not rejected.** The connection
+carries a schema generation, bumped whenever the catalog is invalidated; a
+statement compiled against an older one is recompiled on next use. That is what
+`sqlite3_prepare_v2` does on `SQLITE_SCHEMA`. If it no longer compiles at all,
+the failure is reported and the handle stays registered, so the error is
+repeatable rather than one-shot. The count is observable through
+`Statement::reprepare_count`, mirroring `SQLITE_STMTSTATUS_REPREPARE`.
+
+## Alternatives rejected
+
+**An error that wraps its layer error and implements `source()`.** The
+idiomatic Rust shape, and it would give callers the full chain. Rejected
+because it cannot derive `PartialEq` (so tests substring-match messages
+instead of asserting errors), and because making sixteen engine enums
+`Send + Sync` to satisfy the channel is a large change to satisfy a facade.
+The cost is real and small: the engine's enums barely implement `source()`
+themselves, and the message they format is the diagnostic.
+
+**One result code.** Simpler, and matches what most drivers expose. Rejected
+because the two answer different questions and SQLite itself offers both; the
+one-code version would have had to guess which, and the guess was open.
+
+**Retrying inside a transaction too.** More uniform, and superficially more
+useful. Rejected as unsound: it double-applies. A variant that rolled the whole
+transaction back and asked the caller to replay is a real design, but it
+requires the caller's statements, which the connection does not keep.
+
+**Never retrying, and returning `Busy` for the caller to handle.** Honest, and
+what the type already supports. Rejected because Requirement 5 makes a settable
+timeout a MUST, and because every consumer would then write the same loop —
+which is what the requirement exists to stop.
+
+**Failing a stale statement with a schema error.** Safe, and what SQLite's
+older `sqlite3_prepare()` did. Rejected because it pushes a retry loop onto
+every caller for something the connection can do itself, and `prepare_v2`
+exists precisely because that was the wrong default.
+
+## Consequences
+
+The error type is comparable in tests, which is why the API suites assert
+`Error::ParamCount { expected: 2, found: 1 }` rather than matching on text. No
+`source()` chain is available to consumers; if one is ever needed, it is an
+additive change to a `#[non_exhaustive]` enum.
+
+Busy handling is only exercisable against a *second process*, because two
+connections in one process do not lock against each other at all (POSIX
+`fcntl` is `(process, inode)`-scoped; stock SQLite closes this with
+`unixInodeInfo`, and this crate has no equivalent). The corpus tests use the
+pinned `sqlite3` as the lock holder, which also makes the claim stronger. The
+in-process gap is tracked as a ratchet in
+`tests/unit/api_durability_test.rs::in_process_connections_lock_against_each_other`
+and is a pre-existing engine defect, not a consequence of this decision.
+
+`Statement::reprepare_count` is public API that exists partly to make a claim
+testable ("compilation happened once"). That is acceptable because it is
+SQLite's own counter rather than an invention.

.openspec/adr/0045-a-transaction-holds-the-connection.md

@@ -0,0 +1,96 @@
+# ADR-0045: A transaction holds the connection; other threads wait
+
+**Date:** 2026-09-10
+**Status:** Accepted
+
+## Context
+
+Spec 013 Requirement 4 says statements on a `Connection` are serialized, and
+they are: the worker thread runs one request at a time. But a `Transaction`
+is several statements, and `Connection` is `Clone` precisely so a pool or
+several async tasks can hold it. Per-statement serialization says nothing
+about what happens between them.
+
+The first consumer to run two concurrent catalog commits found three
+interleavings, reported against the facade:
+
+1. Task B's `BEGIN` lands inside task A's open transaction and is refused
+   with "cannot start a transaction within a transaction". Loud, and
+   survivable.
+2. Task B's statements land inside A's transaction and are committed with
+   it — B's work becomes atomic with work it knows nothing about.
+3. **An autocommit write from task C runs inside whichever transaction is
+   open and is rolled back with it.** `execute` returned `Ok(1)`; the row is
+   gone; nothing errored anywhere.
+
+The third is the one that decides this ADR. It is a write that reported
+success being silently discarded, which is the same failure class as two
+connections on one file not locking against each other — except this one is
+reachable through a single `Connection`, which is the object the API tells
+consumers to share.
+
+The consumer worked around it with a mutex around every call and said either
+a fix or a documented caveat would do.
+
+## Decision
+
+`Connection::transaction` takes exclusion for the guard's lifetime. `Shared`
+holds `Mutex<Option<TxnOwner>>` plus a `Condvar`; `transaction_with` claims
+the slot *before* issuing `BEGIN`, and `Transaction`'s commit, rollback and
+`Drop` release it and wake the waiters. Every request passes through
+`Connection::send`, which is the single choke point, so the gate lives there.
+
+Three arms, in order:
+
+- The handle **is** the transaction — `Transaction` holds a `Connection`
+  clone carrying the transaction's token. Proceeds.
+- The handle is on the **thread that opened** the transaction. Proceeds.
+  Holding a `Transaction` and continuing to use the original handle is what
+  a single-threaded caller has always been able to do, and it matches
+  SQLite, where any statement on a connection with an open transaction runs
+  inside it. Blocking here would be a deadlock against oneself.
+- Anyone else waits.
+
+Re-entry from the thread that already holds the transaction returns
+`Error::TransactionActive` rather than waiting: that is a nesting bug, and
+`SAVEPOINT` is out of scope, so there is nothing legitimate to nest. Waiting
+would hide the bug as a hang.
+
+## Alternatives rejected
+
+**Document it and leave the behaviour.** The consumer explicitly offered
+this and it costs one sentence. Rejected because of interleaving 3: a caveat
+does not make a lost write visible, and the guidance it would give — "wrap
+your own mutex around it" — is exactly the code every consumer would then
+write identically. If the correct use of a type is to always hold a lock
+around it, the type should hold the lock.
+
+**Hold a `MutexGuard` in `Transaction`.** The obvious shape, and not
+expressible: `MutexGuard<'a, T>` carries a lifetime and `make check-mvl-limit`
+forbids named lifetime parameters in `src/`. The same constraint that made
+the worker thread the only expressible design (ADR-0041) applies here, which
+is why this is a slot and a condvar rather than a guard.
+
+**Make every statement claim the slot, including a raw `BEGIN` through
+`execute`.** Would close the gap for consumers who write `BEGIN` as SQL
+rather than calling `transaction()`. Rejected because nothing would release
+it: a caller who issues `BEGIN` and then returns early leaves the connection
+wedged for every other thread, with no `Drop` to recover. Stock SQLite offers
+no such protection either. `execute("BEGIN")` therefore stays unguarded, and
+that is a documented limit rather than an oversight.
+
+## Consequences
+
+- A `Transaction` leaked rather than dropped blocks every other thread on
+  that connection, exactly as a leaked `MutexGuard` would. `Drop` is the
+  release, so this requires actively forgetting the value.
+- Re-entry from a *different thread of the same async task* cannot be
+  distinguished from genuine contention and blocks. No API can see task
+  identity; the consumer wraps blocking calls in `spawn_blocking`, so one
+  task holds one thread for the duration, and the thread check covers it in
+  practice.
+- Mixing `transaction()` with a raw `execute("BEGIN")` on another thread is
+  still unguarded, per the rejected alternative above.
+- Throughput under contention drops to one transaction at a time per
+  connection. That is what the consumer already achieves with its own mutex,
+  and a connection is a single worker thread regardless.

.openspec/adr/index.md

@@ -45,3 +45,6 @@ Specs record what the system must do; ADRs record **why it is shaped this way**
 | [0039](0039-value-payloads-are-arc-not-rc.md) | `Value`'s text and blob payloads are `Arc`, not `Rc` | 2026-09-04 |
 | [0040](0040-streaming-execution-with-batch-as-wrapper.md) | One streaming execution primitive, with the batch path as its wrapper | 2026-09-01 |
 | [0041](0041-embedding-api-owns-the-connection-driver-out-of-tree.md) | The embedding API owns the connection; the `sqlx` driver stays out of tree | 2026-08-28 |
+| [0042](0042-rows-changed-counted-by-codegen-flag.md) | Codegen flags the one mutation that is a row change; `None` is not `Some(0)` | 2026-09-04 |
+| [0043](0043-embedding-api-failure-surface.md) | The embedding API's failure surface: a flat error, both result codes, and autocommit-only busy retry | 2026-09-09 |
+| [0045](0045-a-transaction-holds-the-connection.md) | A transaction holds the connection; other threads wait | 2026-09-10 |