Skip to content

Dialect authoring: carry the complete original inventory by digest and page declarations through the read port (#243) - #250

Merged
kikashy merged 4 commits into
mainfrom
dialect-inventory-paging-243
Oct 29, 2026
Merged

kikashy merged 4 commits into
mainfrom
dialect-inventory-paging-243

Conversation

@kikashy

@kikashy kikashy commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Closes #243. Milestone v0.12.0.

What was wrong

v0.11.0 carried the complete original declaration inventory inline in the read snapshot and in every candidate, checkpoint and proposal, each bounded at 4 MiB. A reachable, accepted Desk collection (150 retained original revisions, 30 text fields each, distinct 1,000-byte descriptions) has a 4,947,643-byte inventory, so the factory refused it at the snapshot bound before authoring began.

The contract now (maintainer decision: declarations by digest, paged reads through the read port)

  • Index. The snapshot, candidate, both draft checkpoints, the preview candidate and the final proposal carry researchIndex: per original revision in order, each ordered field as {name, digest} (sha256:<hex> of the exact fieldJson bytes, the repository's existing digest form), plus digest, the SHA-256 of the canonical JSON of revisions. Nothing is dropped, sampled or reordered; unavailable stays null. Host helpers: dialectResearchIndex / DialectResearchIndexOf.
  • Paged read port. readDeclarations(request, signal) / ReadDeclarations(ctx, request), the read port's paged member. A request names id, packKey, the bound collection revision, inventoryDigest, cursor (position in index order) and limit. Every page is checked before use. These refuse in the existing tool-input style: another revision, inventory, id, pack or cursor; an empty page or a declaration out of order (missing declaration); a digest mismatch; a page over its byte or declaration bound; a declaration that is not the named field's. An index whose digest differs from its entries, a malformed index, or one over its bound refuses at the read. A non-empty index without the paged port refuses. No other access is added. Host consumers (preview, review) read declarations with readDialectDeclarations / ReadDialectDeclarations, which apply the same checks.
  • Binding. The candidate fingerprint, draft digests, preview fingerprint and proposal all cover the index. The propose re-read must show the read's inventory digest, and the currentness read after preview must show the candidate's. Otherwise the proposal refuses before retain.
  • Model surface. read_dialect pages the inventory once and shows researchInventory (digest, declaration count, per-revision field count or null). Its schema gains optional cursor (integer 0..1,000,000). With cursor it returns one checked page and nextCursor. Its description and guidance say so. propose_dialect's schema is unchanged.

Bounds (existing ones unchanged)

constant value arithmetic
DialectIndexBytes / indexBytes 2,097,152 Canonical JSON of the index, half the 4 MiB snapshot bound. A field entry costs 95 bytes plus its name, 255 at the 160-byte name limit. The index holds about 8,200 declarations at the longest names and about 16,700 at 30-byte names, each declaration up to 16 KiB. The issue's 150 x 30 inventory takes 1,175,648 bytes at the longest names.
DialectPageBytes / pageBytes 1,048,576 Canonical JSON of one page. Canonical escaping at most doubles a 16 KiB declaration, so a page always holds 31 of the largest.
DialectPageDeclarations / pageDeclarations 1,000 Declarations per page, the existing structural limit.

Versioning

  • Draft checkpoints and review proposals written now have version 2 and carry researchIndex, never researchDeclarations.
  • Version-1 checkpoints written by v0.11.0 still read. dialect-draft.json keeps their shape and adds a reader rule: version 2 requires researchIndex and forbids researchDeclarations, and version 1 forbids researchIndex. checkpoint.version is now enum [1, 2]. The new $defs are researchIndex, indexRevision and indexField, and the new candidate member is researchIndex.
  • The Go and TypeScript candidate types keep researchDeclarations as an optional, read-only member for version-1 records.
  • Public surface change for hosts: Go's DialectSnapshot.ResearchDeclarations is replaced by ResearchIndex plus the ReadDeclarations port. A TypeScript snapshot that still carries inline researchDeclarations is refused with that named reason.
  • dialect-authoring.json adds read.cursor and the port shapes pageRequest, declaration and page.

Files

  • Go: go/context/dialects.go, new go/context/dialects_inventory_test.go, updated dialect tests, go/certification/scenarios_dialects.go (index and pager in the scenario host), regenerated go/contract/contract_gen.go and record_shapes.json, and dialects_shapes_test.go (14 inspection checkpoints).
  • TypeScript: packages/harness/dialects.ts, new dialects-inventory.test.ts, testing/dialects.ts (the test host serves an index and byte-bounded pages), updated dialect tests, contract/record-shapes.json, and scripts/mutation-rows.json.
  • Contract: schemas above, contract/README.md rows, new vectors contract/vectors/dialect-authoring/paging.json, new witnesses in inputs.json (cursor read, page request, page) and retention.json (two version-2 checkpoints), authoring.json expectedFingerprint (recomputed outside the package; the candidate now carries the index), and tests/test_dialect_authoring_contract.py.
  • Docs: new docs/adr/0028-dialect-inventory-by-digest.md, and docs/design/dialect-authoring.md with a new section "Paging original declarations (Dialect factory bounds exclude accepted Desk collections with complete original inventories #243)" for hosts that implement the read port. No earlier ADR owned dialect authoring; its contract lived only in the design document, so ADR-0028 is the first record and records this amendment.

Vectors and tests

  • paging.json is handwritten; its digests come from SHA-256 outside the package. Both languages drive every case:
    • a 3-revision inventory served in three pages succeeds, with the exact page requests listed;
    • a page whose declaration does not match its digest is refused;
    • a page from another collection revision is refused;
    • a page that omits a declaration is refused;
    • a retain against a different inventory digest (changed during preview) is refused before retain;
    • an index over its bound (9 x 1,000 x 160-byte names, 2,296,781 bytes, inside the snapshot bound) is refused.
  • Reproduction: TestDialectInventoryDeskDomainReproduction (Go) and the TypeScript test "reproduces the 150 x 30 x 1,000-byte Desk domain …" build 150 revisions x 30 fields x distinct 1,000-byte descriptions synthetically. The inline inventory exceeds both 4 MiB bounds. On main the same domain refuses with Dialect authoring: the read snapshot exceeds its byte limit. (run against an export of origin/main). Here it is read, paged twice (9,000 declarations), proposed and retained. The snapshot, candidate, draft and proposal stay within their bounds, every served page stays within the page bounds, and the paged reader recovers the exact inventory.
  • Parity: the same handwritten vectors, refusal messages and the authoring fixture's expected fingerprint pass in both languages.

Safeguards and mutants

Each was removed once in a scratch copy, and its test failed at the intended assertion. TypeScript rows are in packages/harness/scripts/mutation-rows.json, origin agent#243, grouped at the end. Go twins were run by hand.

safeguard TS row Go twin
skip the digest check on a page caught (paging vector) caught
accept a page from another revision caught caught
let retain ignore the inventory digest caught caught
drop the page byte bound caught caught
drop the index byte bound caught caught
skip the index digest check caught caught
accept an empty page (missing declaration) caught caught
proposal ignores an inventory change since the read caught caught

Three existing rows changed their find text to the moved code: prior exact read, complete-original-inventory and strict-original-declaration. All three are caught again. The Go twin of complete-original-inventory needs the retain binding off to reach its assertion, because the new binding catches it first.

Review notes

Follow-ups after the clean room on 968ed6a:

  • 6e512e7 declares the new test file in scripts/parity.sh.
  • c71a041 fixes F1 and F4:
    • F1: refusal tests for a page that echoes another cursor, id or packKey, in both languages. There are 3 TypeScript rows plus Go twins, all caught.
    • F4: TypeScript copies the host page first, then checks the copy and uses only the copy. An accessor test pins this, with 1 row, caught.

Host-facing break, for the release notes:

  • Go. DialectSnapshot.ResearchDeclarations is removed.
    • Use ResearchIndex *DialectResearchIndex, built with DialectResearchIndexOf.
    • DialectAuthoringPorts.ReadDeclarations is required whenever the index names a declaration.
    • DialectCandidate gains ResearchIndex. Its ResearchDeclarations is omitempty and used only for version 1.
    • Drafts and proposals are written as version 2.
    • A Go host that builds the struct directly gets a compile error. A Go host that decodes a JSON snapshot still carrying researchDeclarations silently drops it, so the inventory reads as unavailable (F7).
  • TypeScript. A snapshot carrying researchDeclarations is refused by name: "the read snapshot carries an inline researchDeclarations inventory; supply researchIndex and readDeclarations pages instead."
    • Use researchIndex (from dialectResearchIndex) and ports.readDeclarations.
    • The checkpoint and proposal version type is 1|2.
  • Mixed version 1/2 shapes are refused by the schema only (F5). The generated Go codec, the Go native walker and the TypeScript checkRecord accept them, as they already do for the existing stage conditional.
  • Re-encoding a version-1 record through Go's DialectCandidate is not byte-faithful (F6). It adds researchIndex: null and drops researchDeclarations (null and [] alike). No package path re-encodes version-1 records, and the generated contract codec stays byte-preserving.

Checks

Candidate: head 968ed6a301f80fd313f87f77c89999c85f1df058 with no uncommitted diff. Scope: jpdev check --changed origin/main selected all 24 steps. Local toolchains were Go 1.26.5 (the manifest names 1.26.10, which is not installed) and Node 22.

first run (jpdev check --changed origin/main):
PASS generated-docs, record-shapes, contract, local-paths, agent-docs, diff-check, python-tests, mutation-rows, build, release-quality-engine
FAIL packed-smoke   the temporary directory had an ancestor node_modules (machine layout, not the change)
rerun: TMPDIR in a separate system temporary directory (removed afterwards), jpdev check --only packed-smoke
PASS packed-smoke   packed harness: imported 125 supported entrypoints ... from an isolated installed tarball
remaining steps (jpdev check --only ...):
PASS reference, unit-tests (150 files passed, 2 skipped; 3823 tests passed), certify-vercel, certify-codex, certify-deepagents
PASS differential (150 cases), go-module-path, gofmt, go-vet, go-generate, go-test, go-race, harness-check (202 legs, 94 agent checks)
SKIP parity        needs JPDEV_DESK (the Desk checkout); not run
mutation rows: 11 selected rows caught (mutation-check.mjs --row ...); 11 Go twins caught by hand

Self-checks:

  • (a) No boundary path changed.
  • (b) Every added schema member and enum value is carried by a vector. The Python witness tests delete each one.
  • (c) Both readers pass the new vectors.
  • (d) Rows and twins: see above.
  • (e) The refusals are tool-input failures on the existing tool_result path. No new event kind.
  • (f) No execution-checkpoint member was added. Draft checkpoints carry the index through a fresh factory, tested by the checkpoint-resume tests.
  • (g) The design section and ADR were read against the code.

CI owns the exact-head run and the push-to-main mutation table.

🤖 Generated with Claude Code

…clarations (#243)

The complete original declaration inventory no longer travels inline in the
read snapshot, candidate, checkpoints and proposal. They carry a bounded index
instead: per original revision, each ordered field's name and the SHA-256 of
its exact declaration bytes, plus a digest over the canonical index. The
declarations arrive through the read port's paged member, bound to the
collection revision and inventory digest, and every declaration is checked
against its indexed digest and order before use.

- New bounds beside the existing ones: index 2 MiB of canonical JSON,
  pages of at most 1,000 declarations and 1 MiB. Existing bounds unchanged.
- read_dialect shows an inventory summary and pages declarations with
  cursor/nextCursor; propose re-pages the inventory; the currentness read
  after preview must show the candidate's inventory digest before retain.
- Checkpoints and proposals are version 2 with researchIndex; v0.11.0
  version-1 records with inline researchDeclarations still read.
- ADR-0028, design section on paging, paging vectors, the issue's
  150 x 30 x 1,000-byte reproduction in both languages, mutation rows.

Closes #243

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
@kikashy kikashy added this to the v0.12.0 milestone Oct 29, 2026
kikashy and others added 2 commits October 29, 2026 02:32
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
…page (#243)

Review of record F1: one refusal test per echoed member (cursor, id,
packKey) in Go and TypeScript, with mutation rows. F4: TypeScript copies the
host's page before checking it and uses only the copy, so an accessor cannot
change declaration bytes after the checks; a test pins it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Carry needed from base commit 3e0662a.

@kikashy

kikashy commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Clean-room review record: agent PR #250 (issue #243)

Re-read at c71a041: mergeable, 0 HIGH, 0 MEDIUM. F1 and F4 are closed. F2, F3, F5, F6 and F7 are LOW and unchanged, and F5–F7 are declared in the PR body. The delta adds no new exposure. The details are in the last section. The first-round verdict below was for head 968ed6a.

First round, at 968ed6a. Verdict: mergeable, 0 HIGH. On every probe, in both languages, no declaration byte reached a candidate, preview, draft, retained proposal, checkpoint, model request or tool answer without first passing its digest and order check. The one exception is the TypeScript accessor case in F4. It needs a host port that returns a self-changing object, and that is outside the digest's threat model. There is 1 MEDIUM (F1: some page echo checks are pinned by no test) and there are 6 LOW. The bounds hold exactly at their stated numbers. Go and TypeScript count canonical bytes identically. The 150 x 30 x 1,000-byte domain is refused on main at the snapshot bound and retained on the head. All 4 TypeScript PR rows and all 4 Go twins are caught. Of my own mutants, 4 per language: the Go and TypeScript "propose re-pages", the TypeScript count-bound and the Go partial-reader mutants are caught. The cursor and id/packKey echo mutants survive the dialect suites in both languages (F1).

  • Candidate: 968ed6a301f80fd313f87f77c89999c85f1df058 (PR Dialect authoring: carry the complete original inventory by digest and page declarations through the read port (#243) #250 head), with no uncommitted diff. Its merge base and comparison point is origin/main at dd507c45a0068d4c9a31fa0b7d33898694e2f0c9. The clone is at <scratch>/repo, and main is a worktree at <scratch>/main-wt. Probes are in <scratch>/probes/ and mutants in <scratch>/mut-go/.
  • Toolchains: Go 1.26.5 (the manifest names 1.26.10, which is not installed) and Node 22.23.1.
  • Caps:
    • Go ran under ulimit -v 8000000 with timeout. Runs that included certification used the 12 GB cap with Node 22 first on PATH.
    • Vitest and mutation-check.mjs ran under the kit's rsscap.sh (6000 MB for probes, 12000 MB for the mutation run).
    • GOCACHE and TMPDIR were inside the room and were cleaned afterwards.
  • Same-vendor statement: this is a Claude Opus review over a Claude Opus build. It is not a cross-vendor claim. I did not build this change and had no contact with its builder. I posted, pushed and edited nothing outside the room.

Method

  1. Consumer list, derived from the code rather than the PR. I searched origin/main and the head for every use of ResearchDeclarations / researchDeclarations, dialectResearchInventory / researchInventory, the new index and port names, and NewDialectTools / dialectTools.
    • On main, the inline inventory was used in four places only:
      • read validation;
      • the read_dialect answer to the model;
      • the candidate, which carried it into both draft checkpoints, the preview candidate and the proposal;
      • host code (the Go certification scenario and the TypeScript test host).
    • Column research.fieldJson is a separate, per-column declaration on both main and the head, and was never cross-checked against the inventory.
    • On the head, every one of those places now carries only the index.
    • Declarations reach anything only through dialectPage / declarationPage. Its callers are:
      • verifyInventory, at read and again at propose;
      • pageTool, the model's cursor read;
      • the exported host reader ReadDialectDeclarations / readDialectDeclarations.
    • No path reads a partial or sampled set:
      • verifyInventory and the host reader loop to the end of the index;
      • pageTool returns one checked page with nextCursor, by design.
  2. One probe harness in each language, with the same named scenarios (<scratch>/probes/zz_review_probe_test.go and zz-review-probe.test.ts). Each harness drives the exported tools through the PR's own test host, with a pager I wrote that can transform each page. Go writes the shared boundary inventories to inputs.json, and TypeScript reads the same file. I then diffed the two outputs line by line.
  3. Version-1 round trips. I decoded each v1 row of retention.json into Go DialectDraftCheckpoint, on main and on the head. I also put mixed shapes through the Go generated codec, the Go native walker and the TypeScript checkRecord walker.
  4. Mutants.
    • Go: in a scratch worktree, one at a time, running the Dialect tests in ./context and ./certification. Certification needs Node 22 and the 12 GB cap. Without them it fails for environment reasons, so the rows were re-run with both.
    • TypeScript: through scripts/mutation-check.mjs --row, on its own scratch copy. I added my rows to the table in a worktree copy, never in the clone.
  5. The reproduction, run once on each side.

Probes (head results; Go and TypeScript identical except where marked)

probe command head result
baseline 3-revision inventory, pages of 2 go test ./context -run TestReviewProbes / vitest run zz-review-probe.test.ts retained
duplicate declaration within a page same refused: "omitted an indexed declaration or broke its order"
two declarations swapped same refused: order
revision two's typed bytes presented as revision one's typed same refused: "does not match its indexed digest"
right bytes under the wrong revision label same refused: order
port returns more than limit same refused: "exceeds its declaration bound"
page echoes another cursor same refused: "bound to another collection revision or inventory"
port skips one (serves from cursor+1, echoes cursor) same refused: order
port repeats page 0 for every cursor same refused: order
declaration bytes change between page 1 and page 2 same refused: digest
empty page mid-inventory same refused: "omitted an indexed declaration"
null declarations same refused (Go: "omitted…", TS: "page is malformed")
port error on page 2, then a cursor read same read refused with the port error; the cursor read then refused: "read … before paging" (no read was remembered)
cancellation during page 2 same refused (Go context canceled, TS run-cancelled failure)
host index whose digest disagrees with its rows same refused: "index digest does not match its entries"
non-empty index, no paged port same refused: "requires the paged declaration read port"
index and page agree on bytes that are not that field's declaration same refused: "actual original field declaration"
TOCTOU: declarations change after the read, index unchanged same the propose re-page refused: digest
model cursor 0 / 4 / 5 / -1 / 1000001 / "0" / null / 2.5, and paging without a read same 0 and 4 answer one checked page and the correct nextCursor (null at the end); the rest refused
model cursor 1.0, 1e0 same Go refused, TS answered (F3)
index at exactly 2,097,152 canonical bytes / 2,097,153 same, input built in Go and shared retained / refused: "index exceeds its byte limit"
page at exactly 1,048,576 canonical bytes / 1,048,577 (70 declarations) same retained / refused: "page exceeds its byte limit"; the at-bound page is also answered to the model via cursor
60 declarations of 16,000 < each, within the page bound same read retained in both; Go cursor page refused "the result exceeds its byte limit", TS answered (F2)
index digest over non-ASCII, <&>, U+2028, emoji, DEL names; null and empty field lists same Go and TS give the same digest sha256:8594aaa5… and 320 canonical bytes; retained
own accessor property on fieldJson that changes after 4 reads TS only the model saw "…UNCHECKED…" (F4)
Go dialectDeclaration with invalid UTF-8 ad hoc Go test refused
v1 rows decoded into Go DialectDraftCheckpoint and re-encoded ROWS=…retention.json go test -run TestReviewV1RoundTrip on main and head main: 3/12 identical; head: 0/12; head adds researchIndex:null to every v1 row, and drops researchDeclarations null (rows 6, 16) and [] (row 7) (F6)
v2+researchDeclarations, v1+researchIndex, v2 without researchIndex Go codec, Go native walker, TS checkRecord all accepted; only the JSON Schema refuses (F5)
record shapes structural diff of record_shapes.json, main vs head; cmp with the TS copy identical in both languages; only current changed (new $defs, read.cursor, version enum, allOf length); historical and published untouched
contract_gen.go line multiset diff the only removals that are not re-added are gofmt realignment of the registry, the Read/Propose union fields and two comments/member lists; new types are only the index, page and request types
reproduction on main (dd507c4) TestReviewDeskDomainOnMain (the head's domain via main's inline API) inline inventory 5,028,901 bytes; refused: "the read snapshot exceeds its byte limit." The head's own test does not compile on main (no DialectDeclarationPage / DialectResearchIndex)
reproduction on head go test ./context -run TestDialectInventoryDeskDomainReproduction PASS

The canonical-byte arithmetic in the ADR holds:

  • a field entry costs 95 bytes plus its name;
  • about 8,200 declarations fit at 160-byte names;
  • canonical escaping at most doubles a valid JSON declaration, so 31 of the largest fit in a page.

TypeScript measures with utf8Bytes(canonicalJSON(…)) and Go with len(dialectCanonical(…)). Both land on the same exact-bound inputs.

On allocation: the page and the index are each canonicalized once, at roughly three times the host's own data, before the count or shape checks run. The snapshot is already bounded at 4 MiB before the index check. A host can make the factory allocate a multiple of a page it returned, but not beyond what it handed over.

Mutants

# lang mutant result
1 TS (PR row) skip the page digest check caught: paging vector "declaration does not match its digest"
2 TS (PR row) accept a page from another revision caught: "page from another collection revision"
3 TS (PR row) retain ignores the inventory digest caught: "retain against another inventory digest"
4 TS (PR row) drop the index byte bound caught: "index over its bound"
5 Go (PR twin) dialectHash(d.FieldJSON) != want.digest → false caught: TestDialectInventoryPagingVectors/declaration_does_not_match_its_digest
6 Go (PR twin) drop page.Revision != request.Revision caught: …/page_from_another_collection_revision
7 Go (PR twin) post-preview inventory check → false caught: …/retain_against_another_inventory_digest
8 Go (PR twin) index byte bound off caught: TestDialectInventoryPagingVectors and TestDialectInventoryHostRefusals ("index bound missing")
9 Go (own) propose skips verifyInventory caught, but only by the call-sequence assertions (["read","page"]) in TestDialectCompatibilityUnknownDraftAcknowledgment, not by an integrity assertion
10 TS (own) propose skips verifyInventory caught the same way (dialects-compatibility.test.ts, call sequence)
11 Go (own) drop page.Cursor != cursor survived (./context and ./certification both pass)
12 Go (own) drop page.ID / page.PackKey echo checks survived
13 TS (own) drop page.cursor!==cursor survived the dialect selection. The tool reports it "caught" only after the whole-suite rerun, where unrelated tests failed in the scratch copy (providers-reference.test.ts +14); that run is environment noise, not a catch
14 TS (own) drop page.id / page.packKey echo checks survived the dialect selection, same as 13
15 TS (own) drop declarations.length>request.limit caught: "host refusals page over its declaration limit"
16 Go (own) ReadDialectDeclarations stops after the first page caught: paging vector "paged inventory succeeds" and the reproduction

Totals:

  • PR rows: 8 of 8 caught (4 TypeScript rows, 4 Go twins).
  • Own mutants: 4 caught (9, 10, 15, 16) and 4 survived (11 to 14).

Findings

# severity where what evidence
F1 MEDIUM go/context/dialects.go dialectPage; packages/harness/dialects.ts declarationPage The cursor echo check and the id/packKey echo checks on a page are pinned by no test in either language. ADR-0028 and the design doc list them among the page refusals. There is no integrity exposure: declarations are bound by position, by indexed digest and by the pinned revision and inventory-digest echoes. Only the safeguard is unpinned. Fix: add one host-refusal case per member (wrong cursor, wrong id, wrong packKey) to TestDialectInventoryHostRefusals and its TS twin, plus mutation rows. mutants 11 to 14
F2 LOW go/context/dialects.go dialectResult (unchanged) on the new pageTool path Go's json.Marshal escapes <, > and & as six bytes each, so a page within the 1 MiB canonical bound can exceed the 2 MiB tool-result bound. The model's cursor read then refuses in Go while TS (JSON.stringify) answers. It fails closed, and only for markup-heavy declarations on host pages filled to the byte bound. Fix: encode with SetEscapeHTML(false), or measure the result canonically. probe "angle"
F3 LOW Go readTool cursor parsing json.Unmarshal into int refuses 1.0 and 1e0. JSON Schema integer and the TS path (already-parsed arguments) accept them. It fails closed. cursor probes
F4 LOW packages/harness/dialects.ts declarationPage (and the existing validateSnapshot pattern) TS validates the host's page and then structuredClones it. An own accessor property on fieldJson can return checked bytes during the checks and different bytes to the clone, and the model then receives unchecked bytes. Go is immune: strings are immutable and pages are copied by value. This needs host code that deliberately returns a self-changing object, and the host supplies the index too, so the digest (which guards drift) cannot defend against such a host anyway. Rated LOW on that basis; re-grade if host ports are not trusted. Fix: clone first, then validate the clone. probe "ts-accessor-toctou"
F5 LOW contract/schemas/dialect-draft.json; record walkers in both languages The mixed-shape rule (v2 requires researchIndex and forbids researchDeclarations; v1 forbids researchIndex) lives only in the JSON Schema if/then. The Go generated codec, the Go native walker and TS checkRecord all accept v2+researchDeclarations, v1+researchIndex and v2 without researchIndex. This matches the existing schema-only stage conditional, and no production record references dialect-draft.json. Release notes should say "the schema refuses". mixed-shape probe
F6 LOW Go DialectCandidate (ResearchIndex without omitempty; ResearchDeclarations now omitempty) Reading a v1 record into the context type is value-faithful. Re-encoding it is not, and it is worse than on main: it adds "researchIndex":null, which the new v1 schema rule forbids, and it drops researchDeclarations whether null or [], so "unavailable" and "known empty" collapse. No package path re-encodes v1 records, and the generated contract codec stays byte-preserving (TestDialectGeneratedCodecInspection, 14 rows). v1 round-trip probe (main 3/12 identical, head 0/12; rows 6, 7, 16 lose the member)
F7 LOW Go DialectSnapshot The host-facing break is asymmetric. A TS snapshot carrying researchDeclarations is refused with a named reason. A Go host that builds the struct directly gets a compile error, which is loud. But a Go host that decodes a JSON snapshot still carrying researchDeclarations silently drops it, and the inventory reads as unavailable (researchIndex nil). Release notes should say so. code reading; the certification scenario itself decodes such a fixture and then sets the index by hand

For the release notes, the host-facing break exactly:

  • Go:
    • DialectSnapshot.ResearchDeclarations is removed; use ResearchIndex *DialectResearchIndex (build it with DialectResearchIndexOf).
    • DialectAuthoringPorts.ReadDeclarations is required whenever the index names any declaration.
    • DialectCandidate gains ResearchIndex, and its ResearchDeclarations becomes omitempty for v1 only.
    • Draft checkpoints and proposals are written as version 2.
  • TypeScript:
    • DialectSnapshot.researchDeclarations is removed, and a snapshot carrying it is refused with "the read snapshot carries an inline researchDeclarations inventory; supply researchIndex and readDeclarations pages instead."
    • Use researchIndex (from dialectResearchIndex) and ports.readDeclarations.
    • DialectCandidate.researchIndex is new and required in the type; researchDeclarations is optional, for v1 only.
    • The checkpoint and proposal version type is 1|2.
  • Model surface:
    • read_dialect shows researchInventory instead of researchDeclarations.
    • It gains an optional cursor (0..1,000,000).
    • propose_dialect's schema is unchanged.

The maintainer's decision is honoured. The domain is not narrowed: the index covers every original revision and field in order, and nothing is clipped, dropped or sampled. The limits are not raised: the snapshot, candidate, draft and runtime bounds are unchanged, and the new bounds are additional ones.

Not established

  • The parity step against a Desk checkout (JPDEV_DESK); no Desk code was run.
  • The full Go and TS suites, go-race, the TS certification suites, packed-smoke and the exact-head CI run. CI owns these; I ran only targeted tests, my probes and the mutants.
  • A real host storage pager. All probes use handwritten pagers over the PR's test hosts.
  • Allocation was reasoned from the code, not profiled.

Re-read at c71a041

  • Candidate: c71a041f2477b88e9ac3844d53b60329cff77774, with no uncommitted diff. The delta since 968ed6a is two commits:
    • 6e512e7: one line in scripts/parity.sh.
    • c71a041: three Go refusal cases, three TS refusal cases plus one accessor test, four TS mutation rows, and a two-line change in packages/harness/dialects.ts.
  • No Go production code changed.
  • Same caps and toolchains as the first round. Same-vendor statement: this is a Claude Opus review over a Claude Opus build. It is not a cross-vendor claim. I did not build this change and had no contact with its builder. I posted, pushed and edited nothing outside the room.
  • Full suites were not re-run; CI is green on this head.
check command result at c71a041
F1, Go: my mutants 11 and 12, re-applied as cursor echo, id echo, packKey echo, and id+packKey together scratch worktree, go test ./context -run Dialect all 4 caught at TestDialectInventoryHostRefusals/page_echoing_another_{cursor,id,packKey}
F1, TS: my mutants 13 and 14 re-applied, plus the PR's three new echo rows mutation-check.mjs --row … on its scratch copy 5 of 5 caught by the selected dialect tests ("host refusals page echoing another cursor / id / packKey"), not by the whole-suite fallback
F4, TS: the PR's row "checks a copy of the page, not the host object" same caught: the new accessor test
F4: my first-round accessor probe (fieldJson good for 4 reads) vitest run zz-review-probe.test.ts the model gets the checked declaration; the "UNCHECKED" bytes no longer appear. The other 39 probe lines are identical to the first round
F4: deep-copy variants vitest run zz-accessor-r2.test.ts clean, with no unchecked bytes in the answer, for: a getter good only on the first read; one good for 4 reads; a getter on the array element; a getter on declarations; the host mutating the page after returning it; and the host reader readDialectDeclarations under a changing getter. A getter bad on its first read is refused (digest). A throwing getter is refused ("page is malformed")

F4 is closed because copyPage (structuredClone) runs once on the port's return value, before any check. It reads each own enumerable property exactly once and deep-copies nested arrays and objects. Every later read (canonical size, binding echoes, order, digest, declaration shape and the returned copy(declarations)) is of that copy. Both the model path (pageTool, verifyInventory) and the host path (readDialectDeclarations) go through declarationPage.

New exposure: none found.

  • A page that cannot be cloned is now refused as "a declaration page is malformed". That covers functions, Proxies and throwing getters. Class instances lose their prototype accessors, and the copy then fails the key check. All of these fail closed.
  • The clone runs before the byte check, so a host that returns an oversized page costs one more copy of its own data. This is the same order of cost already noted for the canonical pre-check.
  • The parity.sh line only declares the new test file.

Review notes in the PR body, checked against this record:

  • F7 and the Go break are stated accurately: ResearchDeclarations is removed, ResearchIndex and DialectResearchIndexOf replace it, ReadDeclarations is required for a non-empty index, DialectCandidate gains ResearchIndex with ResearchDeclarations omitempty for v1 only, and the version is 2.
  • The TS named refusal is quoted exactly, and the version type 1|2 is right.
  • F5 ("refused by the schema only") and F6 (adds researchIndex: null, drops null/[]) match the probes.
  • Two items of the break are not repeated in the Review notes but are stated elsewhere in the same body:
    • the TS candidate type change (researchIndex added, researchDeclarations optional for v1), in "Versioning";
    • the model surface change (researchInventory replaces researchDeclarations in the read_dialect answer, plus the optional cursor), in "Model surface".
  • Gaps for whoever writes the release notes:
    • F2 and F3 (Go fails closed on markup-heavy pages and on a 1.0/1e0 cursor) are not mentioned. They are LOW and need no release-note entry, but they are open.
    • The body's "Checks" section still names 968ed6a as its candidate (cosmetic).

Totals at this head:

  • F1: 4 Go and 5 TS mutants, all caught.
  • F4: 1 PR row caught, and 8 probe variants clean or refused.
  • Findings now open: F2, F3, F5, F6 and F7, all LOW.

…ging-243

Conflict: packages/harness/scripts/mutation-rows.json, both sides appended
rows at the end; kept both groups (agent#242 rows, then agent#243 rows).
scripts/parity.sh merged cleanly with both new_files entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
@kikashy
kikashy merged commit 9c53071 into main Oct 29, 2026
8 checks passed
@kikashy
kikashy deleted the dialect-inventory-paging-243 branch October 29, 2026 07:58
@github-actions

Copy link
Copy Markdown
Contributor

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

kikashy added a commit that referenced this pull request Oct 29, 2026
Carries #250 (dialect inventory paging). mutation-rows.json keeps main's agent#243 rows, then agent#223's; record shapes, the Go contract types and the event reference are regenerated by their generators.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
kikashy added a commit that referenced this pull request Oct 29, 2026
…nded for every accepted collection (#252) (#256)

* Dialect authoring: a two-level inventory index whose top level is bounded for every accepted collection (#252)

The #250 one-level index was checked whole before any page was read, so an
accepted Desk collection (450 revisions x 30 fields with 64-byte names) had a
2,180,240-byte index and was refused at the 2 MiB cap.

The snapshot, candidate, draft checkpoints, preview and proposal now carry a
top index: per original revision its field count and the SHA-256 and
canonical byte length of its revision field index, plus the whole-inventory
digest. Each revision field index (ordered names and declaration digests) is
read through the read port's new readFieldIndex / ReadFieldIndex member and
checked against its top entry; declarations are paged and checked as before.

Bound proof (ADR-0028 amendment, a Python arithmetic test, and paging.json
bounds rows driven in both languages): every top index the shape admits is
at most 1,089,098 bytes, a Desk collection's at most 284,098; a Desk revision
field index is at most 4,956 bytes and an unescaped 1,000-field one 255,186.
Every envelope cap is unchanged.

The version-2 record shape of #243 is in no release and changes in place;
version-1 records still read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>

* Dialect inventory: pin the field index length check and the revision limit; one blank set in both languages (#252)

Review of record F1: a paging case refuses a field index whose canonical
length differs from its top entry while its digest matches, and a bounds row
refuses 1,001 revisions inside the byte cap. Two mutation rows (agent#252),
Go twins applied by hand.

F2: blank text is one explicit code point set in TypeScript and Go, the union
of String.prototype.trim and strings.TrimSpace, so labels of only U+0085 or
only U+FEFF are refused in both; bounds rows carry both cases and an accepted
label that contains them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>

---------

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dialect factory bounds exclude accepted Desk collections with complete original inventories

1 participant