Skip to content

bug: the file change counter (header offset 24) is never incremented — a long-lived sqlite3 reader keeps serving a stale cache #710

Description

@dpsiderius

Description

The file change counter at header offset 24 is never incremented on a write.
It is preserved rather than zeroed, so a file we write keeps whatever value
it had, and a file we create from scratch keeps its initial one.

It stays consistent with the version-valid-for number at offset 92, which is
why nothing has caught it: PRAGMA integrity_check does not look at either,
and our own reads have no cache to invalidate.

What the counter is for

It is the cache-invalidation signal for another connection. Stock SQLite
reads the change counter when it takes a SHARED lock; if it differs from the
value it saw last time, it discards its page cache and re-reads. Offset 92
holds the change-counter value for which the file-size field at offset 28 is
valid, which is why the two must move together.

So the failure mode is specific: a stock sqlite3 connection that has a
database open, caches pages, then sees our write, and keeps serving its
stale cache
— because as far as it can tell nothing changed. No error, no
corruption on disk, just one reader answering from a version of the file that
no longer exists.

Why it matters

This is an interop defect, and interop is the entire premise of the project.
It cannot be seen by any test that only round-trips through our own code, and
it cannot be seen by integrity_check — it needs a long-lived oracle
connection spanning our write, which no test does today: every corpus test
invokes sqlite3 fresh per assertion, so it re-reads the header every time
and never consults its cache.

Found while auditing the embedding API, where it matters more than it used to:
a consumer keeping a stock sqlite3 reader alongside sqlite-rs writers is a
realistic migration path.

Scope

  • Increment offset 24 on the first write within a transaction (SQLite does it
    once per transaction, not once per page), and set offset 92 to the same
    value when the file-size field is valid.
  • src/header.rs owns both fields; the pager's commit path is where the bump
    belongs.
  • Wrapping: it is a u32 and SQLite lets it wrap. Do not treat wrap as an
    error.

Non-goals

Acceptance Criteria

  • A test with a long-lived sqlite3 process — one that has read the
    database, then observes our write, then reads again — sees the new rows.
    This is the test the bug is invisible without; a fresh sqlite3 per
    assertion cannot detect it
  • Offset 24 increments exactly once per committed transaction, not once
    per modified page
  • Offset 92 equals offset 24 after a commit, so the offset-28 file size
    stays valid
  • A rolled-back transaction leaves both unchanged
  • A file we create from scratch has the same initial values stock sqlite3
    gives a fresh database (byte-compare against an oracle-created empty
    file)
  • Wrapping past u32::MAX is not an error

Complexity

Estimate: small
Reasoning: Two header fields, one increment, in a commit path that already
writes page 1. The estimate is small for the fix and almost entirely about the
test: proving it requires a persistent oracle process rather than the
fire-and-forget sqlite3 invocation the corpus harness uses, so the harness
needs a way to hold a session open across our write.

Refs: #193, #705

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions