Skip to content

jpdev check --changed: run only the check-list steps the changed paths map to - #40

Merged
kikashy merged 2 commits into
mainfrom
check-changed-33
Oct 25, 2026
Merged

kikashy merged 2 commits into
mainfrom
check-changed-33

Conversation

@kikashy

@kikashy kikashy commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What it adds

  • jpdev check --changed [BASE] (bin/jpdev, lib/changed_checks.py, standard library only). The changed paths are those that differ between the merge base of BASE and HEAD and the working tree (committed, staged, unstaged and deleted paths, both sides of a rename), plus untracked files that are not ignored. BASE defaults to the default branch: origin/HEAD, else origin/main, else origin/master. The paths are mapped to check-list steps through the changed rules of jpdev.json (path globs to check names), and only those steps run (passed to the runner as --only).
  • The whole list runs, and the report says why, when a changed path matches no rule (unmapped path X: whole list), when jpdev.json, the manifest or scripts/dev/check.sh changed (selection input changed: X: whole list), when nothing changed, or when there is no mapping. Before the summary block, the report names each step selected with the globs that selected it, and each step not selected.
  • Globs match the whole repository-relative path: * and ? stay within one segment, and a ** segment matches any number of segments. [ ] { } \, a rooted glob, an empty, . or .. segment, a ** that is not a whole segment, a check the manifest does not list, an empty or repeated list and an unknown member are all refused with exit 2 before any step runs, and the message names the member or rule (for example jpdev check --changed: refused: jpdev.json changed[1].checks: no check named 'go-tset' in dev.json). An unresolved BASE, a missing default branch or merge base, and --changed together with --only are refused the same way. --skip and --all still apply.
  • templates/jpdev.schema.json and templates/jpdev.json carry the changed member. generators is now required only when changed is absent (anyOf). lib/generated_docs.py, and its copy scripts/dev/generated_docs.py, accept the member and do not read it. This repository's own jpdev.json maps its documentation paths: docs and top-level Markdown to diff-check, local-paths; the agent docs also to agent-docs, tests; docs/adopting.md and templates/*.md also to tests. Every other path runs the whole list.
  • docs/policy.md has a "Scoped runs" paragraph. The scoped run is allowed for local iteration and on pull requests. The full list is required on the default branch, before a release, and whenever check-selection semantics changed. The stale sentence that named jpdev check --changed: run only the check-list steps the changed paths can affect #33 as separate work is removed. README documents the command and its flags, docs/adopting.md has a "Changed-path check selection" section, and docs/generated-docs.md names the new member.
  • Public surface: a new jpdev check option and its output block, and a new jpdev.json member with its schema. Vectors and documentation for both are changed in this pull request.

Checks

Candidate: head 69d1d94 (this branch, one commit on 5da7aff), no uncommitted diff. The whole list ran once on that head:

scripts/dev/check.sh --all     (python3 = Python 3.12.11 first on PATH, GIT_EDITOR unset, TMPDIR and JPDEV_SCRATCH in session scratch)
== check summary (dev.json) ==
PASS     diff-check               0s
PASS     local-paths              0s
PASS     agent-docs               0s
    sync-agent-docs: CLAUDE.md matches AGENTS.md
    sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS     bash-n                   0s
    bash -n: every script parses
PASS     json                     0s, python <scratch>/py312bin/python3 (Python 3.12.11)
    parsed 11 JSON documents
PASS     shellcheck               4s
    shellcheck: clean
PASS     tests                    73s
    429 passed, 0 failed
== 7 passed, 0 failed, 0 skipped, 0 not run ==

Environment notes. The session's default python3 is 3.8, which the existing generated-docs tests do not support (Path.is_relative_to). CI's 3.12 was put first on PATH for this run. The session also exports GIT_EDITOR, which the existing cleanup tests refuse as an inherited Git setting, so it was unset. On main (5da7aff) under 3.12 with GIT_EDITOR still set, the tests step failed only at clean deletion boundaries hold in disposable fixtures (427 passed, 1 failed). Neither condition touches this change.

Focused runs during development:

  • tests/changed_checks_test.py: 19 tests OK. It covers the glob semantics, and fixture clones of a bare origin driven through bin/jpdev check --changed with the handwritten tests/fixture/changed-dev.json and changed-jpdev.json. The cases are a docs-only change, a mapped Go path, an unmapped untracked path, two rules giving their union, a rename away from an unmapped path, a change to jpdev.json and to dev.json, no mapping, no jpdev.json, no changed path, an explicit BASE with --skip, 13 invalid mappings plus unreadable JSON, an unresolved BASE, no default branch, and --changed with --only. It also checks that the shipped mappings name steps of their manifests, and that the schema accepts the shipped mappings and refuses 16 bad vectors.
  • tests/generated_docs_test.py: 20 tests OK against both helper copies.
  • A scoped run of the candidate on a scratch clone of itself, with a docs/policy.md edit and --changed HEAD, selected diff-check and local-paths (docs/**), and 2 passed. Adding lib/probe.txt printed unmapped path lib/probe.txt: whole list.

Safeguard probes. Each one broke the code once in a scratch copy, then ran the test file. The clone was not changed.

Broken Caught by
a mapped path selects nothing docs-only, Go path, union and BASE tests, at assertEqual(self.ran(out), {...}) (the whole list ran instead)
an unmapped path is ignored unmapped-path and rename tests, at assertIn('unmapped path ...: whole list')
a selection-input change is ignored the selection-input test, at assertIn('selection input changed: jpdev.json: whole list')
rename detection on (--no-renames removed) the rename test
untracked files left out the unmapped-path test
a check name the manifest lacks is accepted the invalid-mapping test, case unknown check (exit 0, wanted 2)
* crosses / test_star_stays_within_one_segment and the union test
the generated-docs checker refuses changed test_change_mapping_member_is_left_to_jpdev_check

Self-checks from AGENTS.md:

  • (a) does not apply: dev.json here has no handover member.
  • (b) holds. Each of the 16 schema constraints this change adds (anyOf, the relaxed required, the changed property, its type and minItems, the rule's type, additionalProperties and required, and for paths and checks their minItems, uniqueItems, item type and pattern) was removed in a scratch copy, and a vector failed each time.
  • (c), (d) and (f) do not apply: there is one reader, no mutation table and no checkpoint.
  • (e) holds. The refusals are tested with empty input (mapping, paths, glob), missing input (checks), malformed JSON, a wrong version and unknown members. No size bound was added.
  • (g) holds: the docs, usage text and module docstring were re-read against the code.

CI runs the lint and tests jobs on this head. No CI watch was started.

Not in this PR

  • CI still runs the whole list on pull requests, both templates/ci-check.yml and this repository's ci.yml. Moving a pull-request job to the scoped run, and vendoring the selection for repositories that run scripts/dev/check.sh without the kit, are separate.
  • templates/check.sh (the vendored runner) is unchanged: --changed is a jpdev check option, and the runner receives the selection as --only.
  • The prompt of jpdev codex run --small still names jpdev check --only.
  • No version bump and no release notes.

Fix round

Head 27bb34e, one more signed-off commit on 69d1d94. These are the findings of the clean-room review record above.

  • 1 (HIGH), submodules. Every diff now passes --ignore-submodules=none, so a submodule hidden by ignore = all or ignore = dirty (in .gitmodules or the git configuration) or by diff.ignoreSubmodules still counts. The new test test_a_submodule_change_counts_whatever_its_ignore_setting has two cases, each with a docs edit, and each runs the whole list with unmapped path third_party/lib: whole list: a committed pointer bump under ignore = all in .gitmodules, and a changed file inside the submodule under ignore = dirty in the clone configuration.
  • 2 (MEDIUM), reverted changes. The changed paths are now the union of three diffs against the merge base: the working tree, the index (--cached) and HEAD. New tests: a Go change staged and then reverted in the working tree, and a Go change committed and then reverted and staged. Both select the Go steps. The README sentence now says what is diffed, as do docs/adopting.md and the module docstring.
  • 3 (MEDIUM), symlinked manifest. The manifest counts as a selection input both as named and as resolved. In the new test, dev.json is a symlink retargeted from manifests/a.json to manifests/b.json under a rule that maps *.json, and the run prints selection input changed: dev.json: whole list.
  • 4 (MEDIUM), the vendored runner. In the new test, an edit to scripts/dev/check.sh under a rule that maps scripts/dev/** prints selection input changed: scripts/dev/check.sh: whole list and runs the whole list.
  • 5 and 6 (LOW). The schema test no longer skips: a standard-library reader of the keywords the schema uses (it fails on any other keyword) reads every vector, and jsonschema reads them as well where it is installed. The jpdev usage text and docs/adopting.md now list every whole-list reason. docs/adopting.md adds that * matches a name that starts with a dot, and that selection sees only what git reports in this clone: assume-unchanged and skip-worktree edits, ignored files and paths outside the clone are not seen, and the full list remains the guard. The version cases of finding 5 ride, as dispositioned.

Safeguard probes. Each one broke the code once in a scratch copy, then ran tests/changed_checks_test.py.

Broken Caught by
--ignore-submodules=none removed both submodule cases, at assertIn('unmapped path third_party/lib: whole list')
the index diff dropped test_a_staged_change_the_worktree_reverted_still_counts, at assertEqual(self.ran(out), {...})
the HEAD diff dropped test_a_commit_the_index_and_worktree_reverted_still_counts, at assertEqual(self.ran(out), {...})
the manifest as named dropped (resolved path only) test_a_symlinked_manifest_retargeted_runs_the_whole_list, at assertIn('selection input changed: dev.json: whole list')
scripts/dev/check.sh dropped from the inputs test_a_change_to_the_vendored_runner_runs_the_whole_list, at assertIn('selection input changed: scripts/dev/check.sh: whole list')

Each of the 16 added schema constraints was also removed in turn, with python3 -S (no jsonschema), and a vector failed every time. tests/changed_checks_test.py now has 24 tests, all passing; DeclarationTests also passes under python3 -S.

The whole list ran once on 27bb34e, with no uncommitted diff and the same environment as before (Python 3.12 first on PATH, GIT_EDITOR unset, scratch TMPDIR and JPDEV_SCRATCH):

scripts/dev/check.sh --all
== check summary (dev.json) ==
PASS     diff-check               0s
PASS     local-paths              0s
PASS     agent-docs               0s
    sync-agent-docs: CLAUDE.md matches AGENTS.md
    sync-agent-docs: CLAUDE.md matches AGENTS.md
PASS     bash-n                   0s
    bash -n: every script parses
PASS     json                     0s, python <scratch>/py312bin/python3 (Python 3.12.11)
    parsed 11 JSON documents
PASS     shellcheck               3s
    shellcheck: clean
PASS     tests                    76s
    429 passed, 0 failed
== 7 passed, 0 failed, 0 skipped, 0 not run ==

CI runs on this head. No CI watch was started.

Closes #33

🤖 Generated with Claude Code

…s map to

`jpdev check --changed [BASE]` takes the paths that differ between the merge
base of BASE (default: the default branch) and HEAD and the working tree,
plus untracked files that are not ignored, and maps them to check-list steps
through the `changed` rules of jpdev.json (path globs to check names). Only
the mapped steps run. A path no rule covers, a change to jpdev.json, the
manifest or scripts/dev/check.sh, no changed path, or no mapping runs the
whole list. A block before the summary names each step selected with the
globs that selected it, or why the whole list runs. An invalid mapping, an
unresolved BASE and --changed with --only are refused with exit 2 before any
step runs.

The jpdev.json schema and template carry the mapping (generators becomes
optional when changed is present), the generated-docs checker accepts the
member, and this repository maps its documentation paths. docs/policy.md
says the scoped run is allowed for local iteration and on pull requests, and
the full list is required on the default branch, before a release and when
check-selection semantics changed.

Closes #33.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikashy

kikashy commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Clean-room review (same vendor): Claude Opus 5.5, effort xhigh, 2026-10-25, reviewed SHA 69d1d94

Same-vendor review at the maintainer's direction; not a cross-vendor claim. A fresh reviewer in a separate clone at the PR head, CI green; probes under .review/ (21 selection scenarios, 27 schema-versus-code cases, seven mutants). The report, condensed to its findings:

  1. HIGH: the full-list fallback can be bypassed through a submodule (lib/changed_checks.py:154-155). With .gitmodules ignore = all or ignore = dirty, or the user's diff.ignoreSubmodules=all, git diff never lists the submodule path, so it cannot fall through as unmapped. Probed: a pointer bump under ignore=all, a dirty submodule under ignore=dirty, and a bump under the user config each selected only docs-lint,whitespace; go-vet and go-test were skipped. Fix: --ignore-submodules=none on the diff, and a fixture test with .gitmodules ignore = all.
  2. MEDIUM: committed or staged changes the worktree no longer has are not counted (lib/changed_checks.py:154 diffs merge-base→worktree only; the claim at README.md:100). Probed: a Go change staged then reverted in the worktree, and one committed on HEAD then reverted in the worktree, each with a docs edit, both selected docs-only. Fix: union git diff --cached MB and git diff MB HEAD with the worktree diff (checked in a scratch copy with the fix for 1: both cases then select the Go steps or the whole list; the tests still pass).
  3. MEDIUM: a symlinked manifest gets past the "manifest changed → whole list" rule (lib/changed_checks.py:168, realpath resolves the link). Fix: list both the unresolved and the resolved path as inputs.
  4. MEDIUM: no test that scripts/dev/check.sh is a selection input (tests/changed_checks_test.py:176); the mutant dropping it passed all 19 tests. Fix: a fixture change to scripts/dev/check.sh asserting selection input changed.
  5. LOW: schema and code disagree outside the mapping ({"version":1} accepted by code, rejected by schema; malformed generators beside a valid changed; version: 1.0 predates the PR).
  6. LOW: docs and usage text omit cases (usage and docs/adopting.md:204 list only "unmapped"; * matches dotfiles; selection sees only what git reports: assume-unchanged, skip-worktree and ignored files are not seen, nor paths outside the clone; the schema test skips silently without jsonschema).

What holds: globs (dirs, **, ?, dotfiles, spaces, newlines); renames and deletions on both sides; untracked files counted; BASE as tag, remote ref, full ref or HEAD; runs from a subdirectory; an unknown check refused by name; CI and the release workflow unchanged (full list); the repository's own mapping sound; no machine-local path or personal name. Six of seven mutants caught. merge: after fixes.

Disposition (the coordinator's)

# Severity Disposition
1 HIGH Fixed in this PR by the same builder: --ignore-submodules=none, fixture test with ignore = all
2 MEDIUM Fixed in this PR (same round): union of the three diffs; README claim corrected
3 MEDIUM Fixed in this PR: unresolved and resolved manifest paths as inputs
4 MEDIUM Fixed in this PR: the check.sh-input test
5 LOW Rides: the version cases are pre-existing or safe-direction; the schema test is made to fail loudly when jsonschema is absent in CI
6 LOW Rides in the same pass: usage and adopting.md list all whole-list reasons; a sentence on dotfiles and on what git does not report

The same reviewer re-reads the fixed head.

…t paths

The review of the first commit found that the whole-list fallback could be
bypassed. The changed paths are now the union of the merge base against the
working tree, against the index and against HEAD, so a change staged or
committed and then reverted in the working tree still counts. Every diff
passes --ignore-submodules=none, so a submodule hidden by an ignore setting
in .gitmodules or in the git configuration still counts. The manifest is a
selection input both as named and as a symlink resolves it.

Fixture tests cover a submodule under ignore = all and ignore = dirty, a
staged and a committed change reverted in the working tree, a retargeted
symlinked manifest and a change to scripts/dev/check.sh. The schema test
reads its vectors with a standard-library reader, so it no longer skips
without the jsonschema package. The usage text and docs/adopting.md list
every whole-list reason; the README states what is diffed; adopting.md says
that * matches dotfiles and that selection sees only what git reports.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kikashy

kikashy commented Oct 25, 2026

Copy link
Copy Markdown
Contributor Author

Re-read (same vendor): Claude Opus 5.5, effort xhigh, 2026-10-25, reviewed SHA 27bb34e

The same reviewer, the review clone moved to the fixed head; CI green (lint, tests, DCO). Condensed:

Findings 1-4 and 6 are fixed: both submodule cases now run the whole list with third_party/lib reported as unmapped (a bump under the user's diff.ignoreSubmodules=all too); a staged or committed Go change reverted in the worktree selects the Go steps; retargeting the dev.json symlink prints "selection input changed" and runs the whole list; the check.sh-input mutant is caught; usage and adopting.md list every whole-list case, the dotfile and git-visibility sentences are there, and the schema test runs without jsonschema. Finding 5 (LOW, schema and code disagree outside changed) is unchanged and rides. New LOW 7: a submodule counts as one path (its gitlink), so a glob for files inside it never matches and the whole list runs; one doc sentence would make that clear.

The union of the three diffs adds paths but never drops one: a file deleted on HEAD but present in the worktree is counted; committed, worktree-only and undone renames run the whole list; jpdev.json changed on HEAD but reverted runs the whole list; an in-progress merge conflict selects docs-only correctly; BASE=HEAD with a staged change selects the Go steps.

Verified: the fix diff read; 24 + the generated-docs tests OK on Python 3.12; 24 selection scenarios, 27 schema cases and 10 union cases re-run; 11 mutants caught (the original seven and the builder's four); the schema reader broken once in two ways, both caught. No machine-local path or personal name. merge: yes.

Disposition (the coordinator's)

Findings 1-4 and 6 closed; 5 and 7 are LOW and ride (7's sentence goes with the next docs change). Merging on the maintainer's word.

@kikashy
kikashy merged commit c122806 into main Oct 25, 2026
3 checks passed
@kikashy
kikashy deleted the check-changed-33 branch October 25, 2026 18:57
@github-actions

Copy link
Copy Markdown

🎉 Thank you for your contribution to Judgment Pack!

If you’d like another task, choose a path that interests you:

kikashy added a commit that referenced this pull request Oct 25, 2026
Both pull requests extend jpdev.json. The resolution keeps both:
- lib/generated_docs.py and scripts/dev/generated_docs.py accept the
  members {version, generators, changed, duplication} and read generators.
- templates/jpdev.schema.json requires version and one of generators,
  changed or duplication; its title names all three.
- lib/changed_checks.py accepts the duplication member (it refused every
  member but version, generators and changed, so `jpdev check --changed`
  would have refused templates/jpdev.json).
- The standard-library schema reader of tests/changed_checks_test.py reads
  `enum`, which the duplication member's languages use.
- README "What is here" names the duplication check and thresholds.

Signed-off-by: kikashy <35789537+kikashy@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jpdev check --changed: run only the check-list steps the changed paths can affect

1 participant