Skip to content

Release: reuse main's exact-commit CI evidence for the history-independent checks, keep the tag-context run for the rest, fail closed (#38) - #44

Merged
kikashy merged 4 commits into
mainfrom
release-evidence-38
Oct 26, 2026
Merged

kikashy merged 4 commits into
mainfrom
release-evidence-38

Conversation

@kikashy

@kikashy kikashy commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Closes #38. Its "Required evidence gate" list is the acceptance contract. Milestone v0.3.0 (tooling). One review round: release admission is a trust boundary.

Summary

A release no longer runs the whole check list twice more at the tag (once from the tag push's own CI run, once from the release's call). It now does two things in order:

  1. Reuses main's evidence for the checks that read the tree. The release takes the evidence main's push CI recorded at the exact release commit for local-paths, agent-docs and diff-check. It does this through the development kit's release evidence gate (devkit v0.10.0, vendored as scripts/dev/release_evidence.py, configured by .github/ci-evidence.json) in a new job, CI evidence at the tag. Any refusal fails that job, names the code and stops the release.
  2. Runs the entries tests again in the tag's context. These tests read the tags and the release history (release_being_made() and the tag listing in tests/test_entries.py), and the release's tag changes both. So the release still runs them at the tag, after the gate: CI at the tag calls the commit's ci.yml with tag-context: true. The checkout still has the whole history and tags, the specification is still at the pin, and no verifier test is skipped. That call is now the only run in the tag's context. ci.yml's push trigger is limited to branches, so a tag push no longer starts a duplicate run that gates nothing.

Every release-specific check stays exactly as it was and in the same order: tag grammar (shell and release_tag_ok), tag names the commit, commit on main, notes, environment reviewer, manifest blob bytes, checksums, attestation, no existing release, draft identity, notes and asset digests, published digests. The required entries status keeps its name (the job now says name: entries explicitly, which gives the same check name). Branch protection, review and publication requirements are unchanged.

Public surface changes

  • ci.yml gains a workflow_call input tag-context (boolean, default false). A push now runs its list in two parts: the tree checks, whose summary block is the evidence log, then HISTORY_CHECKS (entries). A failure in the first part does not stop the second. On a push to main it then runs release_evidence.py capture and uploads ci-evidence-ci-entries-<attempt> (7 days).
  • ci.yml no longer runs on a tag push (push: branches: ["**"]). Branch pushes and pull requests run as before.
  • release.yml gains the evidence job (CI evidence at the tag, contents: read, actions: read). verify (CI at the tag) needs it, and package needs both. The decision record is in the step log and in the release-evidence-<attempt> artifact, uploaded on reuse and on refusal (!cancelled()).
  • New release precondition: main's CI run at the commit must have completed (run-unfinished otherwise), must be under 48 h old, and must have run on the runner image the release job resolves. The tag must be annotated; git tag -s, which docs/releasing.md already prescribes, makes an annotated tag, and v0.1.0 to v0.2.1 are all annotated.

Inventory: three executions at a release commit

Check Main push CI (before the tag) Tag-push CI (before) Release-called CI (before) After this PR
local-paths, agent-docs, diff-check (read the commit's tree only) run run run main: run and recorded. Release: reused from main's evidence at the exact commit
entries (the unittest suite: reads GITHUB_REF's release tag, the tag listing below it, the earlier releases' blobs, and the specification at the pin) run with no release being made and the tags present then run at the tag run at the tag main: run, not recorded as evidence. Release: run once, fresh, at the tag (CI at the tag, tag-context: true)
setup (python-deps, an empty requirements file) run run run main: run. Release: run before entries
tag, notes, environment reviewer, manifest bytes, checksums, attestation, draft/published identity not run not run release.yml, run release.yml, run, unchanged

Main's evidence and the release's tag-context run do not overlap. A test holds the configuration's reused checks plus HISTORY_CHECKS to exactly dev.json's checks, with no check in both. The tag-push run was equivalent to the release's call: same commit, same refs/tags/<tag> ref (so the same release_being_made()), same full-history checkout, same pinned specification, started by the same push. Its result gated nothing. The release's call now covers every check it ran, which is the issue's condition for suppressing tag CI.

Branch CI is not changed. A branch push (whole list, at the head commit) and a pull request run (the --changed selection, at the synthetic merge commit) have different commits and selections, so they are not equivalent executions. A branch without a pull request has only the push run.

Toolchain and inputs. There is one profile, linux, under compare: release-runner. The release job resolves the image (ImageOS, ImageVersion, /etc/os-release) and bash, git, cmp and python3 itself, and they must equal main's. CI runs the checks with setup-python's 3.12 (3.12.15 from the runner's tool cache, not the image's own python3), so the evidence job sets Python up with the same two steps as ci.yml, and a test holds the two to the same text. Recorded inputs: dev.json, the configuration, the gate, ci.yml (which also holds the specification pin), requirements-dev.txt, scripts/dev/check.sh and scripts/sync-agent-docs.sh. Permitted skips: none (no conditional job, no optional check skip). Trusted event: push to main only; this repository does not dispatch CI.

Fresh route (documented in docs/releasing.md)

A refusal stops the run, and nothing runs CI again on its own. The operator picks one route, once:

  1. Re-run all jobs of main's CI run at the commit. This is also the route for run-age and for a hosted image that changed (runtime-identity). For run-unfinished, wait for main's run instead.
  2. When that run has completed, re-run the release run's failed jobs.
  3. If the gate refuses again: fix on main and release a new version.

Live probe (read-only API)

probe.py walks real runs through the vendored gate's decide() with this configuration, using GET calls only. Three things are stubbed:

  • the release tag, since no tag exists at the probed commits: an annotated tag object naming the commit;
  • the runner identity, with this machine standing in for both runners;
  • in the first walk only, the evidence artifact. Main's ci.yml does not capture until this PR merges, so the probe builds the artifact with the gate's own evidence(). It uses the real run's ids, the real job interval, and the real PASS rows of that job's log for the three tree checks.
Walk Commit Live reads Decision
Latest main CI run 40395182182 (push, attempt 1), evidence artifact synthesized 6398ed8 repository and default branch; workflow ci.yml resolved (id 384594220, name CI, active); runs at the commit: 1, applicable 1; run identity; attempt-1 jobs (entries, success, ubuntu-latest, hosted group 0); re-read of the run; re-selection twice reuse (run 40395182182 attempt 1, age 0.92 h)
The same run, no stub for the artifact 6398ed8 the same, then the real artifact listing refuse: artifact-coverage (0 artifacts named ci-evidence-ci-entries-1: main records no evidence before this merges)
PR #43 head (pull_request run 40390714808, branch push run 40390706439) 88cbf53 runs at the commit: 2, applicable 0 refuse: run-missing
The same two runs handed directly to the gate's run identity check 88cbf53 each run read by id run-trigger for both

The first live capture is the first main push after merge. A capture refusal there fails main's entries job visibly. The first live verify is the next release. Locally, capture with this configuration, given the real check.sh --all --skip entries summary of this head, records {agent-docs, diff-check, local-paths: PASS} and the seven input ids.

Tests and mutants

tests/test_release_evidence.py adds 15 tests. They run inside the entries check, both at main and at the tag:

  • The configuration passes the gate's own check_config.
  • Every input is a file of the tree.
  • The workflow is ci.yml by path and name:, with push only and no skipped jobs.
  • Every required job is a job of ci.yml, on its labels, with no matrix and no job-level if.
  • The reused checks plus HISTORY_CHECKS equal dev.json's checks, with none in both.
  • A handwritten check.sh summary block is read by the gate's own coverage(). The three rows are accepted. Adding entries, dropping a check, or a SKIP each refuses.
  • ci.yml's capture and upload run on a push to main after the checks, from the tee'd log.
  • release.yml's gate needs admit, has actions: read, has no continue-on-error, and keeps its record on refusal.
  • The gate job sets up Python as ci.yml does.
  • CI at the tag needs the gate and passes tag-context: true, and package needs both.
  • ci.yml has no tag trigger.
  • The vendored copy: line 2 names v0.10.0, the rest has the git blob id of the kit's lib/release_evidence.py at v0.10.0 (d00e4d48…, checked against the kit's tag), and the four vendored scripts name one kit release.

The kit's 85 gate tests are not copied: the copy is held byte for byte instead.

Mutant (scratch copy of the head, one safeguard broken) Caught by
M01 the configuration's job name drifts from ci.yml test_every_required_job_is_a_job_of_ci_yml_on_its_labels
M02 an input path that does not exist test_every_input_path_is_a_file_of_the_tree
M03 entries added to the reused checks …_dev_json_split_in_two, test_the_evidence_log_is_read_as_the_gate_reads_it
M04 reused checks "all" the same two
M05 the vendored gate changed after line 2 test_the_rest_is_the_kits_file_byte_for_byte
M06 line 2 names another release test_line_two_names_the_kit_release, test_the_vendored_scripts_name_one_kit_release
M07 ci.yml runs on a tag push again test_a_tag_push_starts_no_second_run_at_the_tag
M08 ci.yml's job display name drifts test_every_required_job_is_a_job_of_ci_yml_on_its_labels
M09 the push path drops the history checks test_ci_records_the_evidence_on_a_push_to_main_after_the_checks
M10 capture on every push, not only main the same
M11 HISTORY_CHECKS emptied …_dev_json_split_in_two
M12 the release calls ci.yml for the whole list test_the_tag_context_run_follows_the_gate_and_the_package_waits_for_both
M13 no decision record on a refusal test_the_gate_runs_after_admission_and_keeps_its_record
M14 package no longer needs the gate test_the_tag_context_run_follows_…
M15 the tag-context run no longer needs the gate the same
M16 continue-on-error on the gate step test_the_gate_runs_after_admission_and_keeps_its_record
M17 the gate job without CI's Python test_the_gate_sets_up_the_python_ci_yml_ran_with
M18 the evidence log no longer the checks' output test_ci_records_the_evidence_on_a_push_to_main_after_the_checks

All 18 were caught, and the unmutated head passes. actionlint 1.7.12 on both workflows reports one finding, SC2034 (attempt unused) in the unchanged draft step, which is the same on main.

Measured before, expected after

Durations are from the API, job start to completion.

Release Main push CI entries Tag-push CI entries CI at the tag / entries Admit end to package start
v0.2.1 (run 39175705155) 12 s (run 39175678865) 14 s (run 39175705135) 13 s 34 s
v0.2.0 (run 39070293785) 17 s (run 39070265083) 14 s (run 39070293773) 12 s 17 s

On the latest main run (40395182182), the three tree checks report 0s each, entries 3 s, and the job takes 13 s, mostly checkout and setup.

After this PR, a release has one fewer full CI run (the tag-push run, about 14 s), and the release's call drops three checks that each take under a second. In exchange it adds the gate job, which runs before the tag-context run. The kit measured its earlier gate job at 14–15 s (v0.8.0, v0.9.0). This gate reads one more repository object and sets up Python, so a similar cost is expected, but that is not measured here.

No saving is claimed. Total runner time at a release is expected to stay about the same. The release's own critical path is expected to grow by about the gate job's duration, because the gate runs first, as the brief orders. What changes is that two tag-context runs become one, and an exact-commit decision record now exists. The first real numbers come at the next release. A refusal adds the fresh route's re-run.

Not changed / not verifiable before merge

  • This PR's CI does not exercise capture, because it runs only on a push to main. The first push after merge will.
  • verify is first live at the next release.
  • update-branches.yml is not changed.
  • Dependabot Bump actions/upload-artifact from 7.0.1 to 7.0.2 #34 (upload-artifact 7.0.2) will need a rebase to cover the two new uses of the 7.0.1 pin.

Checks

Head 134100a, jpdev check --changed against origin/main (merge base 6398ed8; jpdev.json changed, so the whole list ran), with .spec/ at the pin:

== check summary (dev.json) ==
handover: boundary 1 paths, shared 2 paths; see AGENTS.md "Before you hand over"
PASS     local-paths              0s
PASS     agent-docs               0s
    sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS     diff-check               0s
PASS     entries                  6s, python .venv/bin/python3 (Python 3.12.11)
    Ran 153 tests in 6.538s
    OK (skipped=3)
== 4 passed, 0 failed, 0 skipped, 0 not run ==

The handover self-check against origin/main passed: the boundary path scripts gains only a new file.

Fix round

This round answers the review record at 134100a (0 HIGH, 2 MEDIUM, 2 LOW) in one commit, 15ff65e.

Finding Fix Safeguard removed in a scratch copy, then caught by
MEDIUM 1: nothing held the three reused checks to the tree ReusedChecksReadTheTree: the dev.json commands of local-paths, agent-docs and diff-check are pinned in the test (shell toolchain, no cwd/requires_env), so editing one means classifying it again. Each command, and the repository script it runs (scripts/sync-agent-docs.sh, which runs no other), is scanned for a read of a tag, the run's ref or the history or .spec (git tag/describe/log/rev-list/for-each-ref/ls-remote/show-ref/cat-file/show/merge-base/fetch..., GITHUB_, refs/tags/, HEAD~). The scan has its own handwritten vectors R1 (diff-check reads git tag --points-at), R1b (local-paths branches on $GITHUB_REF), R1c (pin and command both moved to git describe), R1d (git log in sync-agent-docs.sh), R1e (the script runs another repository script): all caught
MEDIUM 2: fail-closed was held only through needs: test_no_job_after_the_gate_can_run_past_a_refusal: the release's jobs are exactly admit, evidence, verify, package, attest-and-draft and publish. None has a job-level if: or continue-on-error, and each after the gate needs evidence directly or through the jobs it needs R3 (package if: ${{ always() }}), R3b (verify if: ${{ !cancelled() }}), R3c (publish continue-on-error), R3d (attest-and-draft needs admit only): all caught
LOW 3: release_being_made docstring named a tag push now "release.yml calling ci.yml at the tag"
LOW 4: stale "image python 3.12.3" in this body corrected above: both jobs set up 3.12.15 from the runner's tool cache

Mutants this round: 9 of 9 caught, and the unmutated head passes. tests/test_release_evidence.py now has 19 tests.

Head 15ff65e, jpdev check --changed against origin/main, which ran the whole list because jpdev.json changed:

== check summary (dev.json) ==
handover: boundary 1 paths, shared 2 paths; see AGENTS.md "Before you hand over"
PASS     local-paths              0s
PASS     agent-docs               0s
    sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS     diff-check               0s
PASS     entries                  7s, python .venv/bin/python3 (Python 3.12.11)
    Ran 157 tests in 7.000s
    OK (skipped=3)
== 4 passed, 0 failed, 0 skipped, 0 not run ==

🤖 Generated with Claude Code

…ndent checks, keep the tag-context run for the rest, fail closed (#38)

The release no longer runs the whole check list twice more at the tag. It
takes the evidence main's push CI records at the exact release commit for
the checks that read the commit's tree (local-paths, agent-docs,
diff-check), through the development kit's release evidence gate (v0.10.0,
vendored as scripts/dev/release_evidence.py, configured by
.github/ci-evidence.json), and refuses when that evidence is missing,
partial, stale or different. The entries tests read the tags and the
release history, which the release's tag changes, so the release still runs
them at the tag: release.yml calls ci.yml with tag-context: true, after the
gate. That call is the one run in the tag's context; ci.yml's push trigger
is held to branches, so a tag push no longer starts a duplicate.

ci.yml runs a push's list in two parts (the tree checks, whose summary is
the evidence log, then HISTORY_CHECKS) and, on a push to main, records and
uploads the evidence. Every release-specific check is kept as it was and in
the same order. tests/test_release_evidence.py holds the configuration to
ci.yml, dev.json and the tree, the release's wiring, and the vendored copy
to the kit's file.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikashy kikashy added this to the v0.3.0 milestone Oct 26, 2026
@kikashy

kikashy commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Review record: exchange PR #44 (issue #38), head 134100a, base 6398ed8

The PR is correct on every point in the brief. No HIGH: no reused check reads tags or history, the tag-context run keeps full history and the pinned specification, and a refusal stops everything downstream. Two MEDIUMs are mutants the new tests do not catch: a reused check's command is never held to tree-only, and the downstream jobs are never held free of a job-level if:.
Clean-room review, one round. Same-vendor review, not a cross-vendor claim. The vendored gate is byte-identical to the kit's v0.10.0 lib/release_evidence.py.

# Finding Sev Evidence What to do
1 No test checks that the three reused checks stay tree-only. If one of them starts reading a tag, the release still takes it from main's pre-tag run. MEDIUM R1 (diff-check reads git tag --points-at/describe) and R1b (local-paths branches on $GITHUB_REF) both survive the full suite. Pin the three run strings of dev.json in test_release_evidence.py, so any edit means classifying the check again. Also grep scripts/sync-agent-docs.sh for GITHUB_, git (tag|describe|log|rev-list|for-each-ref|ls-remote) and .spec.
2 "Fails closed" is held only through needs:. A job-level if: always() / !cancelled() on a downstream job would get past a refusal. MEDIUM R3 (package with if: ${{ always() }}) and R3b (verify with if: ${{ !cancelled() }}) both survive. Assert there is no job-level if: in verify, package, attest-and-draft or publish, as the test already does for evidence.
3 The release_being_made docstring still names "a tag push" as a run at a tag. ci.yml no longer runs on one. LOW tests/test_entries.py lines 1159-1160 Say "release.yml calling ci.yml at the tag" (rides with the next change there).
4 The PR body says the image's own python3 is 3.12.3. Main and PR CI now run on ubuntu-26.04 (image 20261012.163), and that figure is from the older image. The reasoning still holds: setup-python gives 3.12.15 from the toolcache in both jobs. LOW Logs of runs 40395182182 and 40403723303 Optional edit to the PR body.

(1)-(5) Checks

  • (1) The entries job runs the setup step python-deps (empty requirements; it runs again at the tag) and four checks.
    • Tree-only: local-paths (git grep over tracked files), agent-docs (cmp AGENTS.md CLAUDE.md), diff-check (git diff --check against the empty tree). A grep of the three commands and of sync-agent-docs.sh finds no GITHUB_REF, tag, log, release-history or .spec read. .spec is untracked and gitignored, so git grep and git diff never see it.
    • Depends on tags, history and the pin: entries (the unittest suite). It reads GITHUB_REF (release_being_made), the tag listing and earlier release blobs (release_commits, catalog_history), the pin (VerifierPin) and .spec (the verifier tests). Its form, dialect, link and adoption tests are tree-only, but they run at the tag too.
  • (2) With tag-context: true, the called ci.yml checks out the tag (git checkout refs/tags/vX, as in the v0.2.1 log) with fetch-depth: 0 and all tags, and the specification at 4bec33a.
    • The only skip conditions are .spec present, .git present and GITHUB_ACTIONS; none depends on the input. Locally, with GITHUB_REF=refs/tags/v0.3.0, --only entries gives 153 tests, skipped=1, matching main CI, PR CI and v0.2.1's CI at the tag.
    • capture and upload cannot run at a tag (github.ref check plus the implicit success()).
  • (3) Nothing depended on the tag-push run. Branch protection requires entries and DCO on main (Actions app); there are no rulesets and no tag protection. admit reads no commit status. update-branches.yml is a separate workflow, and its job is skipped off main. name: entries keeps the context: PR Release: reuse main's exact-commit CI evidence for the history-independent checks, keep the tag-context run for the rest, fail closed (#38) #44 shows entries twice, both green.
  • (4) The release path is admit → evidence → verify (needs admit and evidence) → package (needs all three) → attest-and-draft → publish.
    • No job-level if: and no continue-on-error. A refusal exits 1 with its code, and everything downstream is skipped. The record upload runs on !cancelled() with if-no-files-found: error.
    • The old whole-list call is gone: tag-context is hard-coded, and release.yml has no dispatch.
  • (5) Recorded inputs: the config, ci.yml (which holds the pin), dev.json, the gate, requirements-dev.txt, check.sh and sync-agent-docs.sh.
    • jpdev.json and manifest.json are not listed. Neither decides the three checks on a push, and the exact-commit rule binds both: no finding.
    • The gate job's Python setup is the same text as ci.yml's (a test holds this). Main's run used 3.12.15 from the runner toolcache, so the same image gives the same version. The first run refuses only if the image changed (runtime-identity, which has its documented route).
    • 48 h, the same image, an annotated tag and a completed run are all stated in docs/releasing.md.

(6) Live probe (read-only; only the tag stubbed)

  • Main run 40395182182 at 6398ed8: the gate checks pass in order: repository, default branch, workflow 384594220 CI active, 1 run / 1 applicable, identity, success, age 1.08 h, job entries (ubuntu-latest, hosted). It then refuses with artifact-coverage (0 named ci-evidence-ci-entries-1), expected before the first capture. Locally, runtime() refuses with runner-identity, expected off a runner. Nothing unexpected.
  • v0.2.1 commit with its real tag: the tag object is annotated and peels to the commit. Refusal run-age (190 h), expected. The old tag-push run (branch v0.2.1) is neither applicable nor scanned. v0.1.0, v0.2.0 and v0.2.1 all pass tag_object.
  • PR Release: reuse main's exact-commit CI evidence for the history-independent checks, keep the tag-context run for the rest, fail closed (#38) #44 head 134100a: 2 runs, 0 applicable → run-missing. Each run passed to run_identity gives run-trigger.
  • Simulated main push path on the head (with .spec): the tee'd log holds exactly the 3 PASS rows, entries gives 153 OK, and the tree is clean after the tests. capture gives {agent-docs, diff-check, local-paths: PASS} and 7 input ids.

(7) Mutants (a scratch clone for each)

  • Builder's, re-run, all caught: M03, M07, M09, M10, M12, M13, M14, M15, M16, M17 (10 of 18).
  • Mine, caught: R2 (with: tag-context: true dropped) by test_the_tag_context_run_follows_…; R2b (input misspelt, so always empty) by test_ci_runs_the_history_checks_alone_….
  • Mine, survived: R1, R1b, R3, R3b (findings 1 and 2).
  • Mine, survived statically, caught at run time: R4 (fetch-depth: 1 in ci.yml). A shallow clone fails 5 existing entries tests (artifact-history-invalid path), so this is no finding.

(8) Docs and the PR body

  • The preconditions, the dropped trigger, the fresh route and the dependabot Bump actions/upload-artifact from 7.0.1 to 7.0.2 #34 rebase note are all stated. No local paths in the added text.
  • The prerelease limit (until devkit #54) has no effect here: the exchange's grammar has no prerelease (docs/releasing.md lines 78-79), and release.yml refuses one before the gate.

Verified unchanged

  • Every release-specific step, in the same order: grammar ×2, tag names the commit, on main, notes, reviewer, manifest blob plus cmp, checksums, attest, no existing release, draft id, notes and digests, published digests.
  • Branch CI and the PR selection, the required contexts, and update-branches.yml.

merge: yes (no HIGH). Disposition per the kit policy: both MEDIUMs and the two LOWs are fixed in this PR in one round by the builder and confirmed by this reviewer.

… gate (#38 review)

The review of #44 found two safeguards no test held. The three checks the
release takes from main's evidence (local-paths, agent-docs, diff-check)
are now pinned by their dev.json commands, and each command and the
repository script it runs (scripts/sync-agent-docs.sh) is scanned for a
read of a tag, the run's ref, the history or the specification checkout:
editing one means classifying it again. Every release job after the gate
now must need it, directly or through the job it needs, and none may carry
a job-level condition or continue-on-error, so no job can run past a
refusal. The release_being_made docstring no longer names a tag push.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikashy

kikashy commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Confirmation: 15ff65e

One commit on top of 134100a: 2 files, +99/−2, tests and a docstring only (no workflow, config or gate change). In a scratch clone with .spec at the pin, the suite runs 157 tests OK (3 skipped, all keyed on GITHUB_ACTIONS) and test_release_evidence.py runs 19 OK. PR CI at the head: entries ×2 and DCO green.

  • Finding 1 (MEDIUM), fixed. TREE_CHECKS pins the three run strings, the toolchain and the absence of cwd/requires_env/optional. The commands, and every repository script they name, are scanned for tag, ref, history and .spec reads, and a script that runs another script is refused. Re-run: R1 and R1b are now caught; a new R5 (sync-agent-docs.sh reads git describe) is caught too.
  • Finding 2 (MEDIUM), fixed. The job list is fixed to six jobs. No job may have a job-level if: or continue-on-error, and every job after evidence must have it upstream through needs. Re-run: R3 and R3b are now caught; a new R6 (continue-on-error on a publish step) is caught too.
  • Finding 3 (LOW), fixed: the docstring now names release.yml calling ci.yml at the tag.
  • Finding 4 (LOW), fixed: the PR body now says 3.12.15 comes from the runner's tool cache, not the image's own python3.
  • No new finding. A quoted 'if': key would get past the job-level pattern; that is too contrived to count as a finding.

merge: yes (15ff65e). No HIGH; no MEDIUM open.

kikashy and others added 2 commits October 26, 2026 18:06
scripts/dev/release_evidence.py from judgment-pack-devkit v0.11.0 (the
tag grammar with SemVer prereleases, dispatch validation runs).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
…ease in the copy test

check.sh, handover_check.sh and changed_checks.py from judgment-pack-devkit
v0.11.0 (line 2 only); the vendored-copy test records the v0.11.0 gate
file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
@kikashy

kikashy commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Confirmation: fbd0e86

  • The re-vendor itself is correct. After line 2, the gate is byte-identical to the kit's lib/release_evidence.py at v0.11.0 (blob 55fd77db). .github/ci-evidence.json has the same meaning as before: the JSON is equal, and only the array formatting changed. capture is unchanged.
  • Behaviour on this configuration is unchanged:
    • The grammar is plain vX.Y.Z. A tag push runs as publication. A dispatch run is refused with dispatch-not-gated, and release.yml has no dispatch trigger anyway, so exit 0 still means reuse only.
    • Walks give the same results as before: main 6398ed8 (tag stubbed) → artifact-coverage; v0.2.1 with its real tag → run-age; the PR head → run-missing.
    • v0.3.0-rc.1 → tag-syntax, before any API read.
  • The tree-only and fail-closed tests from 15ff65e still pass.
  • BLOCKING (a red required check, not a trust-boundary HIGH): the copy test was not moved with the re-vendor. Locally 3 tests fail, and PR CI shows entries FAILURE twice:
    • test_line_two_names_the_kit_release and test_the_rest_is_the_kits_file_byte_for_byte fail because KIT_RELEASE/KIT_BLOB still name v0.10.0 / d00e4d48.
    • test_the_vendored_scripts_name_one_kit_release fails because check.sh, changed_checks.py and handover_check.sh still say v0.10.0.
    • Fix: set KIT_RELEASE = "v0.11.0" and KIT_BLOB = "55fd77db1b97bf1824a0456c0b8426b3fdf3d6da", and re-vendor the other three scripts at v0.11.0. Between the two kit releases their bodies differ only in check.sh's version line, so only line 2 changes.

merge: no (fbd0e86) until the copy test is moved and entries is green. After that, a re-run of the suite confirms it; no further review is needed.

Confirmation: 510aa06

  • The commit has 4 files, +5/−5, all on line 2 markers and the copy test's two constants. The copy test now names KIT_RELEASE v0.11.0 and KIT_BLOB 55fd77db.
  • The vendored scripts all match the kit at v0.11.0: check.sh equals templates/check.sh exactly, and changed_checks.py and handover_check.sh equal their lib/ files after line 2. The gate is unchanged from fbd0e86.
  • The copy-test file passes 19/19. The whole check list passes (4 checks, entries 157 OK, 3 skips keyed on CI only), and PR CI shows entries ×2 and DCO green. The blocker is resolved.

merge: yes (510aa06).

@kikashy
kikashy merged commit 0b880b1 into main Oct 26, 2026
4 checks passed
@kikashy
kikashy deleted the release-evidence-38 branch October 26, 2026 22:11
@github-actions

Copy link
Copy Markdown
Contributor

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

kikashy added a commit that referenced this pull request Oct 28, 2026
Release notes for Exchange v0.2.2, a tooling-only release: the
development kit's checks and re-vendored scripts, the changed-path
mapping on pull requests, and the release evidence gate (#44). The
presets, the manifest and the specification verifier pin are unchanged
since v0.2.1.

No release is published by this pull request; the signed tag on the
merged commit starts the release workflow, which waits at the `release`
gate.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid duplicate release CI while preserving current tag-history verification

1 participant