Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/ci-evidence.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"inputs": [
"dev.json",
"scripts/dev/check.sh",
"scripts/dev/changed_checks.py",
"lib/ci_evidence.py",
"templates/check.sh",
"lib/generated_docs.py",
Expand Down
25 changes: 22 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@ name: CI

# The check list is dev.json, run by scripts/dev/check.sh (the kit's own vendored runner). The two
# jobs keep their names, which branch protection requires: `lint` runs the cheap steps, `tests` the
# kit's test run.
# kit's test run. A pull request runs, of each job's steps, those `check.sh --changed` selects
# against the base branch (all of them when a path is unmapped or a selection input changed); a push
# to main and a dispatch run every step, and only they record the evidence a release takes.

on:
push:
Expand All @@ -20,17 +22,26 @@ jobs:
name: lint
runs-on: ubuntu-latest
steps:
# The whole history, so that --changed finds the merge base with the base branch.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
# Whitespace; no machine-local path in tracked text; CLAUDE.md is AGENTS.md and the root
# copy starts with the template; every script parses; no block of code is duplicated over the
# thresholds of jpdev.json; the JSON documents parse; shellcheck (the program the runner image
# carries).
- name: The lint steps of dev.json
env:
EVENT: ${{ github.event_name }}
BASE_REF: ${{ github.base_ref }}
run: |
set -o pipefail
scripts/dev/check.sh --all --only diff-check,local-paths,agent-docs,bash-n,duplication,json,shellcheck | tee "${{ runner.temp }}/checks.log"
if [ "$EVENT" = pull_request ]; then
scripts/dev/check.sh --all --changed "origin/$BASE_REF" --skip tests | tee "${{ runner.temp }}/checks.log"
else
scripts/dev/check.sh --all --skip tests | tee "${{ runner.temp }}/checks.log"
fi

- name: Record exact CI inputs and coverage
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
Expand All @@ -50,11 +61,19 @@ jobs:
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: The kit's tests (jpdev check and check.sh on the fixture, the caps, the record gate)
env:
EVENT: ${{ github.event_name }}
BASE_REF: ${{ github.base_ref }}
run: |
set -o pipefail
scripts/dev/check.sh --only tests | tee "${{ runner.temp }}/checks.log"
if [ "$EVENT" = pull_request ]; then
scripts/dev/check.sh --changed "origin/$BASE_REF" --skip diff-check,local-paths,agent-docs,bash-n,duplication,json,shellcheck | tee "${{ runner.temp }}/checks.log"
else
scripts/dev/check.sh --only tests | tee "${{ runner.temp }}/checks.log"
fi

- name: Record exact CI inputs and coverage
if: github.ref == 'refs/heads/main' && github.event_name != 'pull_request'
Expand Down
6 changes: 5 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,10 @@ jobs:
want="# from judgment-pack-devkit ${RELEASE_TAG} (lib/handover_check.sh)"
[ "${line}" = "${want}" ] || { echo "scripts/dev/handover_check.sh line 2 is '${line}'; the tag ${RELEASE_TAG} needs '${want}'" >&2; exit 1; }
cmp -s <(tail -n +2 lib/handover_check.sh) <(tail -n +3 scripts/dev/handover_check.sh) || { echo "scripts/dev/handover_check.sh is not a versioned copy of lib/handover_check.sh" >&2; exit 1; }
line="$(sed -n 2p scripts/dev/changed_checks.py)"
want="# from judgment-pack-devkit ${RELEASE_TAG} (lib/changed_checks.py)"
[ "${line}" = "${want}" ] || { echo "scripts/dev/changed_checks.py line 2 is '${line}'; the tag ${RELEASE_TAG} needs '${want}'" >&2; exit 1; }
cmp -s <(head -n 1 lib/changed_checks.py; tail -n +2 lib/changed_checks.py) <(sed 2d scripts/dev/changed_checks.py) || { echo "scripts/dev/changed_checks.py is not a versioned copy of lib/changed_checks.py" >&2; exit 1; }
echo "version=${version}" >> "${GITHUB_OUTPUT}"

# git archive writes the named paths of the commit, with the commit's time and each file's
Expand Down Expand Up @@ -198,7 +202,7 @@ jobs:
kit="${unpack}/judgment-pack-devkit-${VERSION}"
[ "$("${kit}/bin/jpdev" --version)" = "jpdev ${VERSION}" ] || { echo "the packaged jpdev does not report ${VERSION}" >&2; exit 1; }
"${kit}/bin/jpdev" env > /dev/null
for path in bin/jpdev lib/wrap.sh lib/handover_check.sh templates/check.sh templates/dev.schema.json scripts/dev/handover_check.sh scripts/sync-agent-docs.sh examples/judgment-pack-agent.dev.json LICENSE NOTICE README.md; do
for path in bin/jpdev lib/wrap.sh lib/handover_check.sh templates/check.sh templates/dev.schema.json scripts/dev/handover_check.sh scripts/dev/changed_checks.py scripts/sync-agent-docs.sh examples/judgment-pack-agent.dev.json LICENSE NOTICE README.md; do
test -e "${kit}/${path}" || { echo "the tarball does not hold ${path}" >&2; exit 1; }
done

Expand Down
3 changes: 2 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ summary block, and anything you stopped on or chose not to do.
- `scripts/dev/check.sh` is a copy of `templates/check.sh`; after a change to the template, copy
it again. Line 2 of the template names the kit version (`# from judgment-pack-devkit vX.Y.Z
(templates/check.sh)`) and moves with `JPDEV_VERSION` in `lib/common.sh` at a release; a test
holds the two together.
holds the two together. `scripts/dev/changed_checks.py` is `lib/changed_checks.py` with the
same version on line 2; after a change to the library file, copy it again.
- This file is `templates/AGENTS.md` plus this section. Change a shared rule in the template and
here together, then run `scripts/sync-agent-docs.sh .` for `CLAUDE.md`.
- Machine settings (paths, caps, name lists) never live in this repository. They are read from the
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ summary block, and anything you stopped on or chose not to do.
- `scripts/dev/check.sh` is a copy of `templates/check.sh`; after a change to the template, copy
it again. Line 2 of the template names the kit version (`# from judgment-pack-devkit vX.Y.Z
(templates/check.sh)`) and moves with `JPDEV_VERSION` in `lib/common.sh` at a release; a test
holds the two together.
holds the two together. `scripts/dev/changed_checks.py` is `lib/changed_checks.py` with the
same version on line 2; after a change to the library file, copy it again.
- This file is `templates/AGENTS.md` plus this section. Change a shared rule in the template and
here together, then run `scripts/sync-agent-docs.sh .` for `CLAUDE.md`.
- Machine settings (paths, caps, name lists) never live in this repository. They are read from the
Expand Down
19 changes: 13 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,10 @@ the same name overrides it. `config/env.example` documents each setting with its
working tree (deletions, both sides of a rename, and submodules whatever their `ignore` setting),
plus untracked files that are not ignored. `BASE` defaults to the default branch: `origin/HEAD`,
else `origin/main`, else `origin/master`. The whole list runs when a changed path matches no rule,
when `jpdev.json`, the manifest or `scripts/dev/check.sh` changed, when nothing changed, or when
there is no mapping. A block before the summary names each check selected with the globs that
when `jpdev.json`, the manifest, `scripts/dev/check.sh` or `scripts/dev/changed_checks.py`
changed, when nothing changed, or when there is no mapping. The vendored
`scripts/dev/check.sh --changed [BASE]` makes the same selection with the vendored
`scripts/dev/changed_checks.py`, where the kit is not installed and in CI on pull requests. A block before the summary names each check selected with the globs that
selected it and each check not selected, or why the whole list runs. An invalid mapping, a `BASE` that names no commit, a missing
default branch or merge base, and `--changed` with `--only` are refused with exit 2 before any
check runs.
Expand Down Expand Up @@ -182,6 +184,10 @@ the same name overrides it. `config/env.example` documents each setting with its
command and is ignored; the product term “adversarial” is not in the base list. `codex run` runs
the same check first and does not start the run on a refusal.
- `jpdev env`: the effective settings and where each came from.
- `jpdev --version` prints the release version (`jpdev X.Y.Z`); `jpdev version` adds, when the kit is
a git checkout, the commit it is at (`jpdev X.Y.Z (vX.Y.Z-4-g1f02688)`), which tells a kit at a
release from one past it. An option `jpdev check` does not know, and a refused `--changed`
selection, name the same.

## The duplication check

Expand Down Expand Up @@ -236,8 +242,8 @@ duplication: 1 duplicate pair (at least 70 tokens over at least 6 lines, 214 fil

The kit runs the check on itself with `python3 lib/duplication.py`, over its Python and shell
files at 100 tokens and 10 lines, and leaves out `scripts/dev/**`: those files are the vendored
copies of `templates/check.sh`, `lib/generated_docs.py` and `lib/handover_check.sh`, kept the
same on purpose.
copies of `templates/check.sh`, `lib/generated_docs.py`, `lib/handover_check.sh` and
`lib/changed_checks.py`, kept the same on purpose.

The exit status is 0 when nothing is found or the check is off, 1 when a pair or an ignore marker
problem is found, and 2 when the declaration is refused or the files in scope do not fit in
Expand Down Expand Up @@ -270,8 +276,9 @@ Python steps, and for other steps that invoke `python3`, name the interpreter us
- `templates/dev.json`, `templates/dev.schema.json`: the manifest by example and its JSON Schema.
- `templates/jpdev.json`, `templates/jpdev.schema.json`: the generated-document declaration, the
changed-path mapping and the duplication thresholds by example, and their JSON Schema.
- `templates/check.sh` and `lib/handover_check.sh`: the public checks a repository vendors as
`scripts/dev/check.sh` and `scripts/dev/handover_check.sh`.
- `templates/check.sh`, `lib/handover_check.sh` and `lib/changed_checks.py`: the public checks a
repository vendors as `scripts/dev/check.sh`, `scripts/dev/handover_check.sh` and
`scripts/dev/changed_checks.py`.
- `templates/codex-brief.md`, `templates/codex-review-brief.md`: the build and review brief
skeletons `jpdev codex brief-template` prints.
- `templates/AGENTS.md`, `templates/CLAUDE.md`: the agent instructions (one text, generated from
Expand Down
28 changes: 24 additions & 4 deletions bin/jpdev
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,27 @@ usage: jpdev COMMAND [ARGS]
neutral phrasing; require the handover heading for a build brief
env print the effective settings and where each came from
--version print the version
version print the version and, when the kit is a git checkout, the commit it is at
(git describe), which tells a kit at a release from one past it
USAGE
}

# kit_version prints the version, with the kit checkout's git describe when the kit is its own clone.
# A repository named by the environment (GIT_DIR under a hook, for one) is never taken for the kit's.
kit_version() {
local top described=""
top=$(kit_git rev-parse --show-toplevel 2>/dev/null) || top=""
if [ -n "$top" ] && [ "$top" -ef "$kit" ]; then
described=$(kit_git describe --tags --always 2>/dev/null) || described=""
fi
printf 'jpdev %s%s\n' "$JPDEV_VERSION" "${described:+ ($described)}"
}

kit_git() {
env -u GIT_DIR -u GIT_WORK_TREE -u GIT_COMMON_DIR -u GIT_INDEX_FILE -u GIT_OBJECT_DIRECTORY \
-u GIT_ALTERNATE_OBJECT_DIRECTORIES -u GIT_NAMESPACE git -C "$kit" "$@"
}

need_value() { [ $# -ge 2 ] || jpdev_die "$1 needs a value"; }

repo_root() {
Expand All @@ -85,7 +103,7 @@ cmd_check() {
if [ $# -gt 0 ] && [[ $1 != -* ]]; then base_args=(--base "$1"); shift; fi
;;
-h | --help) usage; exit 0 ;;
*) jpdev_die "check: unknown argument $1" ;;
*) jpdev_die "check: unknown argument $1; this kit is $(kit_version), and \`jpdev check --help\` lists what it takes" ;;
esac
done
[ "$changed" -eq 0 ] || [ "$only" -eq 0 ] || jpdev_die "check: --changed and --only both choose the steps; give one"
Expand All @@ -95,7 +113,8 @@ cmd_check() {
if [ "$changed" -eq 1 ]; then
# The first line is the selection (whole, or only NAME,NAME); the rest is the report.
local selection first
selection=$(python3 -I "$kit/lib/changed_checks.py" --root "$root" --manifest "$manifest" "${base_args[@]}") || exit 2
selection=$(python3 -I "$kit/lib/changed_checks.py" --root "$root" --manifest "$manifest" "${base_args[@]}") ||
jpdev_die "check --changed: refused by this kit, $(kit_version)"
first=${selection%%$'\n'*}
printf '%s\n' "${selection#*$'\n'}"
case $first in
Expand Down Expand Up @@ -666,7 +685,7 @@ codex_run() {
rm -f -- "$dir/msg.txt" "$dir/pr.md"
prompt="The rules are this repository's AGENTS.md; follow them."
if [ "$small" -eq 1 ]; then
prompt+=$'\nThis is a small fix: run only the check-list steps the change touches with `jpdev check --only <steps>`; CI runs the whole list.'
prompt+=$'\nThis is a small fix: run `jpdev check --changed`, which runs only the check-list steps jpdev.json maps the changed paths to (the whole list when a path is unmapped); do not choose the steps yourself. CI checks the pull request the same way and runs the whole list on the default branch.'
fi
prompt+=$'\n'"$(cat "$brief")"
local args=(exec -m "$model" -c "model_reasoning_effort=$effort" --sandbox "$sandbox")
Expand Down Expand Up @@ -1049,7 +1068,8 @@ case ${1:-} in
carry) shift; cmd_carry "$@" ;;
codex) shift; cmd_codex "$@" ;;
env) shift; cmd_env "$@" ;;
--version | version) echo "jpdev $JPDEV_VERSION" ;;
--version) echo "jpdev $JPDEV_VERSION" ;;
version) kit_version ;;
-h | --help | help | "") usage; [ -n "${1:-}" ] || exit 2 ;;
*) echo "jpdev: unknown command '$1'" >&2; usage >&2; exit 2 ;;
esac
34 changes: 27 additions & 7 deletions docs/adopting.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,24 @@ mkdir -p scripts/dev
devkit=path/to/judgment-pack-devkit
cp "$devkit/templates/check.sh" scripts/dev/check.sh
cp "$devkit/scripts/dev/handover_check.sh" scripts/dev/handover_check.sh
cp "$devkit/scripts/dev/changed_checks.py" scripts/dev/changed_checks.py
```

Line 2 of the template, `# from judgment-pack-devkit vX.Y.Z (templates/check.sh)`, records the kit
release the copy came from. A vendored copy keeps that line, so a later reader can tell which release
it is behind. The kit's `scripts/dev/handover_check.sh` similarly vendors `lib/handover_check.sh`
with its release on line 2. Vendor `scripts/dev/check.sh` and `scripts/dev/handover_check.sh`
together, and copy both whole files again when updating them. `jpdev setup` names this pair when
either file is absent.

`scripts/dev/check.sh [--setup] [--only A,B] [--skip A,B] [--all]` runs the manifest with no
caps and prints the summary block. Before a setup or check run, it creates a missing declared
either file is absent. `scripts/dev/changed_checks.py`, the selection code of `--changed`, is
`lib/changed_checks.py` with the release on line 2; vendor it beside `check.sh` and update the two
together (`check.sh --changed` refuses when it is absent, and the CI template uses `--changed` on
pull requests). A runner given an option it does not know names the release on its line 2, and
`jpdev version` prints the installed kit's version with the commit a kit checkout is at.

`scripts/dev/check.sh [--setup] [--only A,B | --changed [BASE]] [--skip A,B] [--all]` runs the
manifest with no caps and prints the summary block. `--changed` selects the checks as
`jpdev check --changed` does (see [changed-path check selection](#changed-path-check-selection)),
with the same rules and the same report, through the vendored `scripts/dev/changed_checks.py`. Before a setup or check run, it creates a missing declared
Python environment with `python3.X` on PATH, then the newest
`~/.pyenv/versions/3.X.*/bin/python3`; it refuses before any step when neither exists. Every step
runs with that environment's `bin` first on PATH, so the manifest's Python setup step installs its
Expand Down Expand Up @@ -100,8 +107,21 @@ regenerates `CLAUDE.md`.
## 4. Add the CI workflow

Copy `templates/ci-check.yml` to `.github/workflows/ci.yml`. It reads the toolchain
versions from `dev.json`, sets the toolchains up, and runs `scripts/dev/check.sh --setup` and
`scripts/dev/check.sh --all`. Its handover job rejects removals from boundary paths unless the pull
versions from `dev.json`, sets the toolchains up, and runs `scripts/dev/check.sh --setup` and the
checks: on a pull request `scripts/dev/check.sh --all --changed origin/<base branch>`, on a push to
the default branch and on a dispatch `scripts/dev/check.sh --all`. The pull request runs the checks
`jpdev.json` maps its changed paths to, and the whole list when a path is unmapped, when the
mapping, the manifest or the runner changed, or when there is no mapping, so a repository without
a `changed` mapping runs the whole list on every pull request as before. The default branch always
runs the whole list, and only that run is the evidence a release takes (see
[trusted CI evidence](ci-evidence.md)); a check a pull request did not select did not pass there,
and the default-branch run is where a mapping that missed an effect shows. This is the
[policy](policy.md#testing)'s split: CI owns every deterministic check, the scoped run is allowed on
pull requests, and the full list is required on the default branch and before a release. The check
job checks out the whole history so that the merge base with the base branch is found. When the
base cannot be resolved (a missing or empty base branch name, or no merge base in the history),
the pull-request job fails with the refusal before any check runs rather than falling back to the
whole list; a push to the default branch runs the whole list whatever the mapping says. Its handover job rejects removals from boundary paths unless the pull
request body contains `Boundary change: yes`, then runs `handover.schema_probe` when present through
the vendored `scripts/dev/handover_check.sh`. Copy
`templates/update-branches.yml` to `.github/workflows/update-branches.yml` to update open branches
Expand Down Expand Up @@ -219,7 +239,7 @@ message that names the member or rule at fault, before any check runs.
`templates/jpdev.schema.json` documents the same shape and `templates/jpdev.json` holds an example.

The whole list runs when a changed path matches no rule, when `jpdev.json`, the manifest (as named
or as a symlink resolves) or `scripts/dev/check.sh` changed, when nothing changed, and when there is
or as a symlink resolves), `scripts/dev/check.sh` or `scripts/dev/changed_checks.py` changed, when nothing changed, and when there is
no mapping (no `jpdev.json`, or one without `changed`). Map only paths whose effect on the checks is
known, and put the checks that read every file (whitespace, path and name checks) in every rule.
Selection sees only what git reports in this clone: an edit hidden by assume-unchanged or
Expand Down
4 changes: 3 additions & 1 deletion docs/ci-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@ PR or another repository is ineligible. Unavailable evidence never becomes succe
No workflow automatically dispatches/retries, and no automatic full CI is disabled.

The unchanged `lint`/`tests` and `contract`/`check` job names retain branch-protection
statuses. Main/PR CI still runs full deterministic checks. Release now verifies
statuses. Main CI still runs the full deterministic checks; PR CI runs the checks its
changed-path mapping selects (`check.sh --changed`, the whole list when a path is unmapped) and
records no evidence. Release now verifies
that the required full deterministic work was completed at the exact source and
compatible inputs, rather than executing those equivalent checks again. Review,
exact-head CI and publication approval requirements remain.
Expand Down
Loading
Loading