Skip to content

bug: SELECT ... FROM sqlite_master does not compile — the catalog is invisible to the SELECT path #707

Description

@dpsiderius

Description

SELECT ... FROM sqlite_master does not compile:

SELECT name FROM sqlite_master
  -> cannot compile statement: unsupported: no such table: sqlite_master

The catalog is decoded and available internally — read_schema is what every
statement compiles against — but it is not reachable as a table from the
SELECT path, because resolve_from_table_schema does not resolve it.

PRAGMA table_info is not an alternative: the introspection pragmas live in
the CLI binary per ADR-0029 and are explicitly out of the embedding API's
scope (spec 013's non-goals defer the PRAGMA catalogue to plan.md V7).

Why it matters

Reported by the SQE consumer against the embedding API. Every SQLite consumer
that needs to know what is in a database reaches for sqlite_master — it is
how you discover tables, and it is how you discover indexes, which nothing
else on the API exposes at all.

Connection::table_names() was added as a workaround and reads the decoded
catalog directly, so consumers are not blocked on table discovery. There is no
workaround for indexes short of parsing the DDL text of each table, which the
consumer explicitly asked not to have to do:

"We use table_names() instead; an index list on the API would let us drop
that dependency on DDL shape."

Scope

Two routes, and the ticket should pick one deliberately rather than doing both
by accident:

  1. Make sqlite_master a resolvable table. Correct in the SQLite sense —
    it is a real b-tree at page 1 with a known five-column shape
    (type, name, tbl_name, rootpage, sql), so a SELECT over it is
    an ordinary table scan. Also gives sqlite_schema (the modern alias) and
    makes WHERE type = 'index' work for free.
  2. An index list on the APIConnection::indexes(table) or similar,
    beside table_names(), reading the decoded TableSchema::indexes that
    already exists.

(1) subsumes (2) for consumers willing to write SQL, and is the answer that
does not grow the API surface. (2) is much smaller and unblocks the consumer
immediately. They are not exclusive.

Non-goals

  • The rest of the introspection pragmas — plan.md V7.
  • sqlite_stat1 as a queryable table (it is already read internally for
    planning).
  • Writing to sqlite_master through SELECT-adjacent paths.

Acceptance Criteria

  • SELECT type, name FROM sqlite_master ORDER BY name returns the same
    rows as the pinned 3.53.4 oracle for the same database, including
    indexes and the sql text
  • SELECT name FROM sqlite_master WHERE type = 'index' works — the index
    discovery case the consumer actually needs
  • sqlite_schema resolves as the same table
  • An empty database returns zero rows rather than failing
  • Connection::table_names() keeps working, and its doc comment stops
    describing itself as a workaround for this
  • Corpus test diffing both against the oracle; spec 013 Requirement 3's
    scoping note about the catalog is amended

Complexity

Estimate: medium
Reasoning: Route (2) alone is small — the data is already on
TableSchema. Route (1) means teaching the FROM resolver about a table whose
schema is not itself in the catalog it reads, which is a small
chicken-and-egg in resolve_from_table_schema, plus deciding whether the
five columns are synthesised or read from a hardcoded TableSchema. The
oracle diff is the easy part.

Refs: 013/Req-3, #705, ADR-0029

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions