Skip to content

Define dialect data quality and typed research results - #241

Merged
kikashy merged 5 commits into
mainfrom
dialect-170-169
Oct 22, 2026
Merged

kikashy merged 5 commits into
mainfrom
dialect-170-169

Conversation

@kikashy

@kikashy kikashy commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Closes #170. Closes #169.

What it adds

  • Declares six closed per-target conversion kinds, their explicit rules and recorded failures.
  • Declares precedence, agreement and first-present reconciliation without implicit last-writer-wins.
  • Surfaces missing, wrong-type, out-of-domain and conflict findings as unknown facts and records every transformation in preparation lineage.
  • Adds the typed research results contract, explicit observation states, versioned acquisition-source profiles, resolved execution settings and retained bytes for offline replay.
  • Updates RFC 0018, the dialect companion, both verifier implementations, the changelog and shared hand-written conformance cases.

Checks

== check summary (dev.json) ==
handover: boundary 2 paths, shared 3 paths; see AGENTS.md "Before you hand over"
PASS local-paths 0s
PASS agent-docs 0s
sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS diff-check 0s
PASS repository-tests 73s
Ran 232 tests in 72.820s
OK
PASS policy-prototype 1s
Ran 38 tests in 1.110s
OK
PASS policy-contract 2s
Ran 13 tests in 2.427s
OK
PASS policy-official-draft 2s
Ran 16 tests in 2.226s
OK
== 7 passed, 0 failed, 0 skipped, 0 not run ==

Not in this PR

Runner implementation remains tracked by Runner #86, and gateway acquisition capabilities remain tracked by Gateway #275.

A Codex session wrote this change.

kikashy and others added 3 commits October 22, 2026 15:23
Add closed per-target conversions, explicit multi-source reconciliation,
quality findings, and preparation lineage so dialects own deterministic data
quality. Define typed research result contracts, versioned acquisition-source
profiles, retained execution evidence, and offline replay rules. Keep the RFC,
companion text, both readers, and hand-written conformance cases aligned.

Co-Authored-By: Codex <noreply@openai.com>
Signed-off-by: kikashy <kikashy@hotmail.com>
…entries

Signed-off-by: kikashy <kikashy@hotmail.com>
@kikashy

kikashy commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Review round 1 (OpenAI gpt-5.6-sol, effort medium, 2026-10-23), reviewed SHA 336b354

Clean-room clone at the head, probes under .review/; CI green before the round (the head since carried past main, 25606ba, with the same content). The report verbatim:

HIGH — tools/dialect_verify.py:2477: readers disagree on date width. For MM/DD/YYYY with "1/2/2026", Python accepts 2026-01-02, while TypeScript’s exact-width regex at tools/dialect-verify-ts/src/prepare.ts:120 records invalid-encoding. Fix Python to enforce the declared lexical form before calendar parsing and add an adversarial vector.

HIGH — tools/dialect-verify-ts/src/prepare.ts:84: decimal-from-number diverges for exponent literals. For 0.01e3, Python produces "10"; TypeScript constructs "0010", then records wrong-type and omits the fact. Fix and specify one exact exponent-expansion/zero-normalization algorithm; add exponent-shift and negative-zero vectors.

HIGH — spec/dialect.md:1066: decimal-from-text does not define valid thousands-separator placement. Both readers currently erase every occurrence, admitting "1,,2" as "12", while another text-conforming reader could reject it. Define the grouping grammar—or explicitly define unconditional replacement—and vector malformed grouping.

HIGH — tools/dialect_verify.py:1914: typed results admits legacy found and not-found states; TypeScript does likewise at tools/dialect-verify-ts/src/forms.ts:1415. Both probes were accepted although §4.3 closes typed results to present, missing, conflicting, inaccessible, and stale. Branch status validation by contract spelling and add rejection vectors.

HIGH — spec/dialect.md:1094: the text says every value outside its declared type records wrong-type, but line 1099 and both readers treat null as valid for every quality type and write it into facts (tools/dialect_verify.py:2487). A text-only implementation can legitimately omit it as unknown instead. State one outcome explicitly; to preserve the typed target contract, record wrong-type, omit the target, and vector it.

MEDIUM — tests/test_dialect.py:2731: the research “replay” vector only calls form checking and copy_claim; it never enters the public verifier or constructs a preparation. retainedBytes is only base64-validated at tools/dialect_verify.py:1982, so changing it to unrelated valid bytes leaves acceptance and results unchanged. Add a complete runnable verifier vector with expected preparation/lineage and exercise it through both verifier entry points.

LOW — rfcs/0018-dialect.md:953: the amendment record calls execution evidence optional, while §4.3 and both readers require execution for results. Say “required for results, optional for legacy fields.”

Verified

  • Reviewed all 11 changed files at 336b354a1bc341251a57b486859b7a1e6e784cdc.
  • git diff --check origin/main...HEAD passed.
  • Python changed-file suite: 82 tests passed.
  • Hand probes covered five conversions and all reconciliation modes; the supplied precedence, first-present, agree, and conflict answers matched their vectors.
  • Digest and lineage integration is present in both readers; last-writer-wins is rejected.
  • No normative text requires a model or web call during replay.
  • TypeScript was inspected for equivalent checks, but its suite could not be rerun because the available Node is v10.19.0 and lacks node --test; CI was already reported passing.

merge: after fixes — 336b354

Disposition (the coordinator's)

# Finding Severity Disposition
1 The two readers disagree on date width (1/2/2026 under MM/DD/YYYY) HIGH Fix in this pull request by the same builder: the declared lexical form enforced before calendar parsing in both readers; a vector
2 decimal-from-number diverges on exponent literals (0.01e3) HIGH Fix in-PR: one exact exponent-expansion and zero-normalization algorithm in the text, both readers; exponent-shift and negative-zero vectors
3 decimal-from-text leaves thousands-separator placement undefined (1,,2 admitted) HIGH Fix in-PR: the grouping grammar defined in the text, both readers; a malformed-grouping vector
4 Typed results admit the legacy found/not-found states HIGH Fix in-PR: status validated by the contract spelling in both readers; rejection vectors
5 null outside a declared type: the text says wrong-type, the readers write it as valid HIGH Fix in-PR: one outcome stated, wrong-type recorded and the target omitted, both readers; a vector
6 The research replay vector never enters the verifier; retainedBytes only base64-checked MEDIUM Fixed in the same push: a complete runnable verifier vector with expected preparation, lineage and execution record; changing the retained bytes changes the result
7 The amendment record calls execution evidence optional for results LOW Fixed in the same push

The same reviewer re-reads the fixed head. Same vendor as the builder, not a cross-vendor claim.

kikashy and others added 2 commits October 22, 2026 15:46
Make both dialect readers enforce exact date widths, decimal expansion and
grouping, typed research states, and null quality findings identically.

Commit retained research bytes and resolved settings to preparation lineage so
public replay detects changed bytes, and add shared positive and refusal vectors
for every corrected behavior.

Co-Authored-By: Codex <noreply@openai.com>
Signed-off-by: kikashy <kikashy@hotmail.com>
Signed-off-by: kikashy <kikashy@hotmail.com>
@kikashy

kikashy commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Re-read after the fix (OpenAI gpt-5.6-sol, effort medium, 2026-10-23), reviewed SHA e51da60

The same reviewer, clean-room clone at the fixed head with Node 22 for the TypeScript verifier, probes under .review/; CI green. The report verbatim:

Findings: none.

Verified

  • All five HIGH fixes align across normative text, Python, TypeScript, and shared vectors.
  • Scratch reversions failed date-width, exponent negative-zero, and malformed-grouping as expected.
  • Typed results reject legacy states; null produces wrong-type.
  • Both public verifiers accept retained bytes and refuse the changed-byte twin with preparation-differs.
  • The amendment now requires execution evidence for results.
  • Checks: Python 83/83 passed; Node 22 fix-focused TypeScript tests passed.

merge: yes
reviewed SHA: e51da60fde9e887173a6e6d59ff69160d1c544df

Disposition: all five HIGHs, the MEDIUM and the LOW are fixed; no further round. Merged on the standing rule. Same vendor as the builder, not a cross-vendor claim.

@kikashy
kikashy merged commit 0786b88 into main Oct 22, 2026
6 checks passed
@kikashy
kikashy deleted the dialect-170-169 branch October 22, 2026 19:56
@github-actions

Copy link
Copy Markdown
Contributor

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant