Skip to content

Public shape: contributor files, the release workflow with attestations, the kit on itself - #4

Merged
kikashy merged 2 commits into
mainfrom
public-shape
Oct 20, 2026
Merged

kikashy merged 2 commits into
mainfrom
public-shape

Conversation

@kikashy

@kikashy kikashy commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Closes #2. Closes #3.

The repository is public. This brings it to the shape of judgment-pack-agent, adapted to a bash kit, and fixes the two open issues. Version stays 0.1.0.

What it adds

  • Contributor files: CONTRIBUTING.md (running tests/run.sh and shellcheck, the DCO sign-off, one pull request per issue, the agent docs), CODE_OF_CONDUCT.md (the same text), SECURITY.md (the same reporting path, scope adapted: a manifest is code), NOTICE, .gitattributes, .gitignore, .github/PULL_REQUEST_TEMPLATE.md, .github/dependabot.yml (github-actions only), and thank-merged-contributors.yml copied as is.
  • The kit on itself: dev.json (steps diff-check, local-paths, agent-docs, bash-n, json, shellcheck, tests), scripts/dev/check.sh (a copy of the template), AGENTS.md (the template plus a "This repository" section) and CLAUDE.md generated from it. ci.yml keeps its lint and tests jobs, which branch protection requires, and runs them through scripts/dev/check.sh --only ...; it also gains workflow_call. The json step is the old "JSON documents parse" lint step, moved into the manifest.
  • README: a "Public" paragraph, install from a release or a clone, and the release and verification pointers. docs/releasing.md is new.
  • The venv on the PATH for every step when the manifest declares one; the runner template carries its version #3: lib/wrap.sh puts the manifest's venv first on the PATH for every step, whatever its toolchain (the toolchain still decides the cap). A kit test runs a node, a go, a shell and a python step that each import a module installed in the venv, and a run without the venv fails all four. templates/check.sh line 2 is # from judgment-pack-devkit v0.1.0 (templates/check.sh); a test holds it to jpdev --version, and the release workflow asserts it; docs/adopting.md says a vendored copy keeps the line.
  • The go toolchain takes a version in dev.json, so a workflow's Go pin has one source #2: the go toolchain takes an optional version (1.26 or 1.26.5) in dev.schema.json. templates/ci-check.yml reads it and passes it to setup-go as go-version, falling back to go-version-file when the manifest names none. jpdev setup and jpdev check select a Go that matches it (the go on PATH, else sdk/goX.Y.Z/bin/go under the new setting JPDEV_GO_SDK_DIR, default ~/sdk) and say so when none is installed, then run with the go on PATH. examples/judgment-pack-agent.dev.json names 1.26.5; the agent repository's own manifest is a change in that repository.

Public surface: the manifest schema (toolchains.go.version), a new setting (JPDEV_GO_SDK_DIR), the PATH a step sees, and line 2 of templates/check.sh. Tests and documentation changed with them.

Release workflow

release.yml runs on a v* tag (vMAJOR.MINOR.PATCH) and is modelled on the agent repository's, without the Go module tag and the mutation-table gate.

Job What it does
admit The tag is in the grammar, annotated, names the commit the run was started for, and that commit is on main; docs/releases/<tag>.md exists, is not empty and ends its first line with the tag; the release environment has a required reviewer.
verify Calls the tagged commit's own ci.yml (lint, tests).
package bin/jpdev --version, line 2 of templates/check.sh and the tag agree, and scripts/dev/check.sh is the template. git archive of bin/ lib/ templates/ config/ scripts/ docs/ README.md LICENSE NOTICE into judgment-pack-devkit-<version>.tgz (gzip -n, so the bytes are reproducible), checksums.txt, one run of the unpacked kit, the files uploaded as an artifact.
attest-and-draft The tag still names the commit and no release exists under it; both files are attested with actions/attest-build-provenance; a draft release is created with the notes from docs/releases/<tag>.md and its id recorded.
publish Behind the release environment (the gate: a maintainer approves). The tag is compared once more; the draft is read by id and must hold the run's notes, title and asset digests; it is published by that id; the published assets are read again and must match.

docs/releasing.md describes the steps, the gate, what to do when a run fails, and how to verify a release: gh attestation verify <file> --repo ... --signer-workflow Judgment-Pack/judgment-pack-devkit/.github/workflows/release.yml --source-ref refs/tags/<tag> --source-digest <commit> --format json. The release environment exists with a required reviewer. The workflow has not been run: actionlint is clean, and the package steps were run by hand on the commit (the tarball is reproducible and unpacks to a kit that reports its version).

Checks

scripts/dev/check.sh --all from the clone root, once. shellcheck 0.10.0, the native binary (release tarball), in the CI form shellcheck -x -P ".:SCRIPTDIR" ....

== check summary (dev.json) ==
PASS     diff-check               0s
PASS     local-paths              0s
PASS     agent-docs               0s
    sync-agent-docs: CLAUDE.md matches AGENTS.md
    sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS     bash-n                   0s
    bash -n: every script parses
PASS     json                     0s
    parsed 5 JSON documents
PASS     shellcheck               1s
    shellcheck: clean
PASS     tests                    20s
    102 passed, 0 failed
== 7 passed, 0 failed, 0 skipped, 0 not run ==

Also: actionlint (native, with shellcheck on PATH, under lib/rsscap.sh 6000 600) reports nothing; git diff --check is clean; no /home/ or /tmp/ path and no personal name in tracked text. Mutation checks on a scratch copy: restricting the venv to python steps fails the four venv tests; removing the Go selection fails six; a stale line 2 fails two; a prefix match of the Go version (1.7 matching go1.77.5) fails one.

Not in this PR

  • The DCO required status context: the maintainer re-adds it once the DCO app covers the repository.
  • The next version's release notes and version bump (docs/releases/, JPDEV_VERSION, line 2 of the template): the maintainer's, per docs/releasing.md. The v0.1.0 tag predates the workflow and has no release.
  • Turning on immutable releases in the repository settings (the maintainer's).

🤖 Generated with Claude Code

kikashy and others added 2 commits October 20, 2026 12:14
…ersion

Closes #2. Closes #3.

lib/wrap.sh put the manifest's virtual environment on the PATH only for python
steps, so a node, go or shell step that ran a python script used the system
interpreter under jpdev check. When the manifest declares a venv, every step now
runs with it first on the PATH; the toolchain still decides the cap. A kit test
runs a step of each toolchain that imports a module installed in the venv.

templates/check.sh line 2 is now `# from judgment-pack-devkit v0.1.0
(templates/check.sh)`, so a vendored copy records the kit release it came from.
A test holds it to `jpdev --version` and scripts/dev/check.sh to the template;
docs/adopting.md says a vendored copy keeps the line.

dev.schema.json gives the go toolchain an optional `version` (1.26 or 1.26.5).
templates/ci-check.yml reads it with setup-go's go-version and falls back to
go-version-file when the manifest names none. jpdev check and jpdev setup select
an installed Go that matches it (the go on PATH, else sdk/goX.Y.Z/bin/go under
the new JPDEV_GO_SDK_DIR) and say so when none is installed, then run with the
go on PATH. The agent repository's example manifest names go 1.26.5.

The tests unset what jpdev check exports to its steps, so they run under it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
…ns, the kit on itself

The repository is public. This brings it to the shape of
judgment-pack-agent, adapted to a bash kit:

- CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, NOTICE, .gitattributes,
  .gitignore, the pull request template, dependabot for github-actions, and the
  thank-merged-contributors workflow.
- release.yml: on a v tag, admit the tag, run ci.yml at the tag, package jpdev
  as judgment-pack-devkit-<version>.tgz with checksums.txt, attest both files,
  draft the release from docs/releases/<tag>.md and publish it behind the
  release environment. docs/releasing.md describes the steps, the gate and how
  to verify an attestation.
- The kit on itself: dev.json, scripts/dev/check.sh (a copy of the template),
  AGENTS.md and CLAUDE.md. ci.yml keeps the lint and tests jobs and runs them
  through scripts/dev/check.sh --only.
- README: a Public paragraph, install from a release or a clone, the release
  and verification pointers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Brian Jin <35789537+kikashy@users.noreply.github.com>
@kikashy

kikashy commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Narrow review record (same vendor, not a cross-vendor claim)

Reviewer: a clean-room Claude Sonnet agent, read-only; the kit tests once, four probes. Verdict: mergeable — head bf5826563fce9b69ccaa0f1df9616b20d0f53ef3. Verbatim report follows.


Review of devkit PR 4, head bf58265. No HIGH and no MEDIUM findings; two LOW.

Findings (most severe first):

  1. LOW, .github/workflows/release.yml line 171 and docs/releasing.md lines 11-14. The tarball holds bin lib templates config scripts docs README.md LICENSE NOTICE, as listed. It leaves out examples/, but the shipped README.md line 123 and docs/adopting.md lines 73, 88 and 90 send the reader to examples/judgment-pack-agent.dev.json. From an installed release that path does not exist. Fix: add examples to the archive paths (the workflow, the doc, and the "does not hold" loop at line 187), or reword the two docs to point at the repository. Confirmed by reading and by listing the tarball.
  2. LOW, release.yml lines 65-71 against the brief's "annotated signed tag". The workflow checks only that the tag is annotated, not that it is signed. docs/releasing.md line 70 says so ("the workflow does not check it"), and the agent repo's workflow does the same. Not a defect; if a signature gate is wanted, that is a separate change. Confirmed by reading.

Answers to the questions:

  1. The venv on the PATH for every step when the manifest declares one; the runner template carries its version #3 (venv): the probe python3 -c 'import sys; print(sys.prefix)' under node, go, python and shell steps printed the manifest's .venv for all four. With the venv line removed from the manifest, all four printed the system pyenv prefix, so a manifest without a venv is unchanged. The go step's ulimit -v still printed 12000000. The node step has no ulimit; its cap is the RSS watcher in wrap.sh, which the diff leaves alone (read). templates/check.sh line 2 reads # from judgment-pack-devkit v0.1.0 (templates/check.sh). release.yml lines 146-157 compare it to # from judgment-pack-devkit ${RELEASE_TAG} (templates/check.sh), require bin/jpdev --version to equal jpdev <version>, and require scripts/dev/check.sh to be byte-identical to the template. The step runs in package and the job fails if they differ, so the assertion is enforced.
  2. The go toolchain takes a version in dev.json, so a workflow's Go pin has one source #2 (Go version): dev.schema.json adds only version on go, with the pattern ^[0-9]+\.[0-9]+(\.[0-9]+)?$. validate_manifest.py adds the same pattern for runs without jsonschema. templates/ci-check.yml has two setup-go steps: go-version from the manifest when it is set, go-version-file when it is not. With go.version 1.99, jpdev check printed "the manifest names go 1.99 and no matching Go is installed (the go on PATH is go1.26.5 ...); the go steps run with the go on PATH". The jpdev setup wording is covered by tests/run.sh line 215. The setup command itself needs an origin remote, so I did not run it by hand.
  3. Release workflow (read in full, actionlint clean):
    • The tag is held to a strict grammar, must be annotated, must name GITHUB_SHA by ls-remote peel, must be an ancestor of origin/main, and needs a non-empty notes file whose first line ends with the tag. The release environment must have a required reviewer.
    • verify calls ./.github/workflows/ci.yml, which has workflow_call.
    • Packaging uses git archive into dist/ and writes checksums.txt for the tarball alone. Local build of the same command: top-level entries are bin config docs lib scripts templates LICENSE NOTICE README.md, exactly the list.
    • attest-build-provenance v4.2.2 is pinned to SHA 4d10147...; gh api confirms that SHA is the v4.2.2 tag. Its subject paths are the tarball and checksums.txt.
    • The draft is created with --notes-file from the notes file.
    • publish is behind environment: release. I confirmed with gh api that the environment exists with one required_reviewers rule.
    • The only token is GITHUB_TOKEN, with no repository secrets. permissions: {} at the top; admit has contents and actions read; verify and package have contents read; attest-and-draft has contents write, id-token, attestations and artifact-metadata write, as in the agent repo; publish has contents write.
    • docs/releasing.md matches the workflow. Its verify form uses --repo, --signer-workflow, --source-ref and --source-digest, for each of the two files.
  4. The kit on itself: dev.json has diff-check, local-paths, agent-docs, bash-n, json, shellcheck and tests. The CI jobs are still named lint and tests and run scripts/dev/check.sh. AGENTS.md is byte-identical to CLAUDE.md. The agent-docs step also compares the head of AGENTS.md to templates/AGENTS.md, and the "This repository" section follows it.
  5. Contributor files: all are present and adapted to bash (no npm or Go wording, shellcheck named as the native program). dependabot.yml lists github-actions only. thank-merged-contributors.yml is identical to the agent repo's. No personal names and no /home or /tmp paths in tracked text (grep). NOTICE names "The Judgment Pack contributors", as accepted.
  6. Hygiene: tests/run.sh gave 102 passed, 0 failed, including the record-gate refusals, the CI skip rule and the shell timeout tests. shellcheck 0.10.0 (a native binary from an earlier scratch download) in the CI form -x -P ".:SCRIPTDIR" over all the script globs was clean. actionlint was clean. git diff --check against the empty tree was clean. jpdev --version prints jpdev 0.1.0. I made no changes to the repository; my probe work was in scratch directories, and I started no background processes.

mergeable — head bf58265


Disposition (LOW never a resume on its own)

# Severity Disposition
1 LOW Fixed by the maintainer in the v0.2.0 release prep: examples added to the archive, the doc and the loop.
2 LOW Accepted as the agent repo's convention; a signature gate is a separate change if wanted.

Merged on green CI; no second round.

@kikashy
kikashy merged commit 8c515c3 into main Oct 20, 2026
3 checks passed
@kikashy
kikashy deleted the public-shape branch October 20, 2026 16:19
@github-actions

Copy link
Copy Markdown

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

@kikashy kikashy mentioned this pull request Oct 20, 2026
kikashy added a commit that referenced this pull request Oct 20, 2026
The version to 0.2.0 (lib/common.sh), the runner template's and the kit's own
vendored line-2 version, examples/ in the release tarball (the workflow, its
unpack check and docs/releasing.md, from PR #4's review), and the notes
docs/releases/v0.2.0.md.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant