Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,8 @@ the same name overrides it. `config/env.example` documents each setting with its
lines of a failure. The exit status is the number of failures.
- `jpdev mutants --row TEXT [--row TEXT]...`: runs the manifest's `mutants` command on a scratch
copy of the clone (`.git` left out, `node_modules` linked) and prints caught or not caught per
row.
row. Clone-local scratch, temporary storage and the copy destination are excluded; their source
parents and siblings are retained.
- `jpdev record post (--pr N | --issue N) --file FILE [--repo OWNER/NAME] [--dry-run]`: reduces
the scratch and home paths to `<scratch>` and `<home>`, refuses to post when any local path
remains, a line matches the name patterns, or the name-pattern file is missing or holds a
Expand All @@ -106,6 +107,19 @@ the same name overrides it. `config/env.example` documents each setting with its
`git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`,
so the push uses the `gh` login and no stored credential. It refuses `--force`, `-f` and a
`+` refspec.
- `jpdev clean [--root DIR]... [--older-than DURATION] [--min-transcript-bytes N] [--yes]`: dry-run cleanup of explicit
session disposal seals; no roots means no deletion. Enroll an empty session root with
`jpdev clean init --root DIR --session TOKEN`, then attest completed disposable paths with
`jpdev clean seal --root DIR --session TOKEN --path REL --kind KIND` (worktree, review, scratch,
transcript or cache). Scratch also needs `--base BASE --removed-worktree OLD`; shared caches need
`--base BASE`. The default age is seven days (`7d`); durations use integer s/m/h/d/w units.
Transcripts require at least 10485760 logical regular-file bytes (10 MiB); use
`--min-transcript-bytes N` to change this inclusive, transcript-only floor, or `0` to disable it.
Bare or saved Git directories and inherited Git evidence redirection are refused.
`.jpdev-clean-keep` holds exact relative reserves. JSON Lines reports candidates, sizes, reasons,
outcomes and totals; refusals exit nonzero. Linux host process visibility, disposal inventory,
Git/remote/PR evidence and all deletion checks remain mandatory with `--yes`. See
[Scratch and cleanup](docs/policy.md#scratch-and-cleanup) for setup, formats and limitations.
- `jpdev worktree add BASE BRANCH [DIR]`, `jpdev worktree list BASE`, and `jpdev worktree remove
DIR [--force]`: manage sibling worktrees. Removal refuses dirty or unpushed work unless forced.
- `jpdev carry --pr N --model M --effort low|medium|high [--dir CLONE] [--worktree BASE]`: clone
Expand Down Expand Up @@ -136,7 +150,8 @@ the same name overrides it. `config/env.example` documents each setting with its
asked for was done. `--sandbox danger-full-access` is an explicit option only. Worktree runs
share caches below `BASE/.jpdev-shared`; `--small` enforces the manifest handover paths and line
bound after the run; `--resume` continues the named thread. Every detected thread id is written
to `.jpdev/output/thread`. A directory passed to `--add` adds every non-ignored file below it,
to `.jpdev/output/thread`. The builder always receives an existing `TMPDIR` beneath its effective
`JPDEV_SCRATCH`, replacing any inherited temporary directory. A directory passed to `--add` adds every non-ignored file below it,
except the handover files and caches, and is refused with its path when nothing remains to add.
- `jpdev codex review --brief FILE --model M --effort E --dir CLONE [--pr N | --issue N]
[--repo OWNER/NAME] [--out FILE] [--sandbox read-only|workspace-write] [--network]`: checks the brief and
Expand Down
31 changes: 28 additions & 3 deletions bin/jpdev
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,13 @@ usage: jpdev COMMAND [ARGS]
name-pattern file, then post the file
push [BRANCH] push BRANCH (default: the current branch) to origin with the gh credential
helper and no stored credential; refuses --force and -f
clean [--root DIR]... [--older-than DURATION] [--min-transcript-bytes N] [--yes]
inspect explicit session disposal seals; dry run by default
transcripts default to at least 10485760 logical bytes; 0 disables the floor
clean init --root DIR --session TOKEN
clean seal --root DIR --session TOKEN --path REL --kind KIND
[--base DIR] [--removed-worktree DIR]
enroll an empty session root, then seal session-created disposable state
worktree add BASE BRANCH [DIR]
worktree list BASE
worktree remove DIR [--force]
Expand Down Expand Up @@ -188,13 +195,28 @@ cmd_mutants() {
copy=$JPDEV_SCRATCH/mutants/$(jpdev_clone_key "$root")-$$
mkdir -p "$copy"
trap 'rm -rf -- "${copy:?}"' EXIT
rsync -a --exclude .git --exclude node_modules --exclude .venv "$root/" "$copy/" || jpdev_die "mutants: the scratch copy failed"
local found rel
# A builder's scratch and TMPDIR can be below the clone. Never copy generated state
# or the destination back into itself; preserve their parents and ordinary source siblings.
local exclude=(--exclude .git --exclude node_modules --exclude .venv) prune=(-path "$root/.git")
local generated generated_real escaped rel
for generated in "$copy" "$JPDEV_SCRATCH" "${TMPDIR:-}"; do
[ -d "$generated" ] || continue
generated_real=$(cd "$generated" && pwd -P) || jpdev_die "mutants: cannot resolve generated directory"
if [ "$generated_real" != "$root" ] && [[ $generated_real = "$root/"* ]]; then
rel=${generated_real#"$root"/}
escaped=${rel//\\/\\\\}; escaped=${escaped//\*/\\*}; escaped=${escaped//\?/\\?}; escaped=${escaped//\[/\\[}
exclude+=(--exclude "/$escaped/")
escaped=${generated_real//\\/\\\\}; escaped=${escaped//\*/\\*}; escaped=${escaped//\?/\\?}; escaped=${escaped//\[/\\[}
prune+=(-o -path "$escaped")
fi
done
rsync -a "${exclude[@]}" "$root/" "$copy/" || jpdev_die "mutants: the scratch copy failed"
local found
while IFS= read -r found; do
rel=${found#"$root"/}
mkdir -p "$copy/$(dirname "$rel")"
ln -s "$found" "$copy/$rel"
done < <(find "$root" -name node_modules -type d -not -path '*/node_modules/*' -not -path "$root/.git/*" -prune -print)
done < <(find "$root" \( "${prune[@]}" \) -prune -o -name node_modules -type d -prune -print)
local args="" row
for row in "${rows[@]}"; do args+=$(printf ' --row %q' "$row"); done
local out=$copy/.jpdev-mutants.out
Expand Down Expand Up @@ -654,6 +676,8 @@ codex_run() {
[ -n "${GOFLAGS:-}" ] || export GOFLAGS=$codex_goflags
[ -n "${npm_config_cache:-}" ] || export npm_config_cache=$codex_npm_cache
[ -n "${XDG_CACHE_HOME:-}" ] || export XDG_CACHE_HOME=$codex_xdg_cache
export TMPDIR=$codex_scratch/tmp
mkdir -p "$TMPDIR" || exit 2
codex "${args[@]}" < /dev/null > "$log" 2>&1
)
local status=$?
Expand Down Expand Up @@ -998,6 +1022,7 @@ case ${1:-} in
mutants) shift; cmd_mutants "$@" ;;
record) shift; cmd_record "$@" ;;
push) shift; cmd_push "$@" ;;
clean) shift; exec python3 -I "$kit/lib/clean.py" "$@" ;;
worktree) shift; cmd_worktree "$@" ;;
carry) shift; cmd_carry "$@" ;;
codex) shift; cmd_codex "$@" ;;
Expand Down
2 changes: 2 additions & 0 deletions config/env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
# `jpdev env` prints the effective values and where each came from.

# Where jpdev keeps its logs, scratch copies for `jpdev mutants`, and Go build caches.
# Builder TMPDIR is always $JPDEV_SCRATCH/tmp; an inherited TMPDIR is replaced.
# Cleanup roots are explicit CLI arguments, never inferred from this setting.
#JPDEV_SCRATCH="${XDG_STATE_HOME:-$HOME/.local/state}/jpdev"

# The machine lock every Go step holds, so two sessions never run Go at once.
Expand Down
2 changes: 1 addition & 1 deletion dev.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@
{"name": "bash-n", "run": "set -eu; for file in bin/* lib/*.sh templates/*.sh scripts/*.sh scripts/dev/*.sh tests/*.sh tests/fixture/*.sh; do bash -n \"$file\"; done; echo 'bash -n: every script parses'", "toolchain": "shell", "summary": "^bash -n:"},
{"name": "json", "run": "python3 -I -c 'import glob, json; files = sorted(glob.glob(\"*.json\") + glob.glob(\"templates/*.json\") + glob.glob(\"examples/*.json\") + glob.glob(\"tests/fixture/*.json\")); [json.load(open(f, encoding=\"utf-8\")) for f in files]; print(\"parsed %d JSON documents\" % len(files))'", "toolchain": "shell", "summary": "^parsed "},
{"name": "shellcheck", "run": "shellcheck -x -P \".:SCRIPTDIR\" bin/* lib/*.sh templates/*.sh scripts/*.sh scripts/dev/*.sh tests/*.sh tests/fixture/*.sh && echo 'shellcheck: clean'", "toolchain": "shell", "summary": "^shellcheck:"},
{"name": "tests", "run": "tests/run.sh", "toolchain": "shell", "summary": "^[0-9]+ passed, [0-9]+ failed", "timeout": 900}
{"name": "tests", "run": "tests/run.sh", "toolchain": "shell", "summary": "^(Failed cases:|[0-9]+ passed, [0-9]+ failed)", "timeout": 900}
]
}
122 changes: 121 additions & 1 deletion docs/policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ Before handover, a builder runs the self-checks in AGENTS.md on its own head. It
touched check steps while iterating and runs the whole check list exactly once at the end. CI must
be green before any review begins. `jpdev codex run` gives Codex clone-local scratch, lock, Go and
npm caches and an XDG cache when those variables are empty, and creates their directories under
the clone.
the clone. Builder `TMPDIR` is always created under its effective scratch, including when an
external temporary directory was inherited.

After a fix round that closed only MEDIUM findings, the review re-read may use the cheaper tier.
After a HIGH finding, the re-read uses the same tier as the review of record. A HIGH is fixed in the
Expand Down Expand Up @@ -135,3 +136,122 @@ verified before the release is announced.
Tracked files, commit messages, pull request bodies and issue comments carry no
machine-local paths and no personal names. A person is referred to by role or by handle where
a reference is needed. Plain, factual wording: what changed, what was checked, what was not.

## Scratch and cleanup

A session removes only state it created. Use `jpdev clean` after the pull request it built or
reviewed merges, after release, and when disk use exceeds 85%. Never remove another session's
data, a live checkout, or an operator reserve. Operator approval authorizes one narrow
`jpdev clean --root DIR --older-than DURATION --yes` invocation; all tool checks still apply.
Inspect the dry run first. Approval does not transfer ownership of existing state.

Before creating disposable state, initialize an empty, session-created root with
`jpdev clean init --root DIR --session TOKEN`. TOKEN is 8–80 letters, digits, underscores or
hyphens, beginning with a letter or digit; choose a unique session prefix. Init creates DIR
if its parent exists, and refuses a nonempty directory. Put the session's clone, worktrees,
review rooms, logs and scratch beneath it. Configure `JPDEV_SCRATCH` and other output locations
there as needed. The wrapper creates builder `TMPDIR` beneath its effective scratch, replacing
an inherited temporary directory. Temporary files stay in scratch, never loose in the system
temporary directory; temporary directories carry the session prefix.

After the session finishes writing a disposable path, explicitly attest its ownership and
contents with `jpdev clean seal --root DIR --session TOKEN --path REL --kind KIND`. This is an
operator/session assertion that every included file, including untracked, ignored and review
evidence, was created by this session and is disposable at the requested milestone. Never seal
someone else's files. Sealing historical unmarked directories is forbidden; init cannot adopt
their nonempty root. Tools do not automatically seal or infer ownership from a name or UID.
A changed path needs a new explicit seal after its evidence has been assessed.

Kinds are `worktree`, `review` (including reviewer clones), `transcript` (a log directory
or an individual regular log/transcript file), `scratch`, and `cache`. Scratch for a removed
worktree also requires `--base BASE --removed-worktree OLD`; shared base caches require
`--base BASE`. BASE and OLD are absolute in the stored seal; supply all relevant roots explicitly
to cleanup. A whole worktree seal may include its own scratch, locks and caches. Seal separate
scratch/lock/cache directories with `scratch` when they outlive that worktree. Base
`.jpdev-shared` contents are eligible only once no linked worktree directory remains. Unknown,
inaccessible, symlinked or out-of-root worktree registration evidence causes refusal. Stale Git
registration entries for verified absent paths beneath authorized, traversable parents are
retained and do not count as active users. Cleanup does not prune Git administrative records or
remove branch refs: a broad prune could affect another session.

`jpdev clean [--root DIR]... [--older-than DURATION] [--min-transcript-bytes N] [--yes]` discovers only disposal seals,
never recursively guesses candidates from directory names. There are no implicit roots; no flags
means no deletion. Missing roots are reported safely. It defaults to a dry run and seven days;
DURATION is a nonnegative integer followed by `s`, `m`, `h`, `d` or `w`. Age is measured from the
newest seal timestamp, candidate directory/file mtime and inventoried content mtime. Thus a new
seal must age even if its logs are old. Transcripts/logs require at least 10485760 bytes (10 MiB)
by default. `--min-transcript-bytes N` sets an inclusive, nonnegative integer floor; `0` disables
it. The floor applies only to the `transcript` kind and counts logical regular-file bytes,
summing all regular files for a directory; directory sizes do not count. Other kinds are unaffected.
Paths are root-relative, exact, and cannot contain empty, dot or parent components.
Overlapping candidate seals are refused. The ownership metadata and keep file are protected.

Place exact root-relative reserves in `DIR/.jpdev-clean-keep`, one per line; blank lines and
`#` comments are allowed. A reserve protects itself, descendants, and any candidate containing it.
Invalid or symlinked keep files cause refusal. Roots, the invoking checkout, home, base checkouts
and any candidate containing an explicitly authorized root are protected. Keep all live checkouts
and operator reserves in this file even when their names look disposable.

Cleanup requires the recorded inode identities and complete SHA-256/stat inventory to match,
including all ignored and untracked files. A clean tracked diff is insufficient. Git worktrees
must have no tracked or staged changes and no unpublished commits reachable from HEAD or any
local ref, including tags, stashes and other branches. Git metadata must lie beneath an explicitly
authorized root. Publication is checked against current `git ls-remote --refs origin`, without
fetching. Locally unavailable remote objects cause refusal. Attached branches require a verified
origin upstream containing HEAD and a GitHub PR with matching branch and HEAD in CLOSED or
MERGED state. Detached worktrees and detached reviewer clones use verified remote reachability
without a PR. Reviewer clones with attached branches use the same upstream and PR checks.
A reviewer clone with any registered linked worktree remains protected. Bare repositories and
saved Git metadata directories at the candidate root or nested within it are unsupported and
refused, including partial HEAD/index/commondir layouts. Only a verified candidate's canonical
`.git` metadata directory is allowed. Assume-unchanged and skip-worktree index flags are refused,
even with an explicit disposal seal: they can hide changed tracked contents.

Inherited `GIT_*` settings are refused except `GIT_OPTIONAL_LOCKS`, `GIT_TERMINAL_PROMPT`,
`GIT_PAGER` and `GIT_CEILING_DIRECTORIES`. Cleanup forces noninteractive, read-only Git commands
and disables the pager; the ceiling only bounds discovery. Index, repository, object, namespace,
shallow-history, executable and config redirection are therefore refused, including unknown
Git settings. System/global Git config and replacement objects are disabled for verification;
external diff and text conversion are disabled for tracked-diff evidence. Authentication that
depends on ambient Git config can consequently be unavailable and cause refusal. Git, remote and PR
evidence must remain available; an unavailable tool, network or
authentication causes refusal. `gh` is needed only for attached branches.

Deletion requires Linux procfs with the initial host PID namespace (Linux initial namespace inode
4026531836), the host mount namespace and root view, and readable process evidence for every
visible process. Processes in another mount namespace or root view cause refusal because path
aliases cannot establish inactivity. A container, sandbox, PID namespace, hidden process view or permission restriction
cannot establish host inactivity and is refused. Process cwd, root, executable, open descriptors,
mappings, command arguments and declared scratch/cache environment paths are checked. Live
processes are reported by PID and never stopped. Symlinks, special files, nested repositories,
mount crossings and changed ownership are refused. The tool requires directory descriptors,
`O_NOFOLLOW` and POSIX locking; unsupported platforms fail closed.

Before removal the tool repeats inspection, moves the candidate to a private sibling quarantine
name, and rechecks its full inventory, Git state, keeps, root and parent identities and process
activity. Cache worktree users and scratch associated-worktree absence are rechecked after
quarantine and before each removal, including the candidate directory itself. Removal uses directory
descriptors, never recursive path-following deletion. Each
entry's identity and inventory is checked during removal. External linked-worktree refs, HEAD
and index are rechecked during removal as well. Changes or failures stop that candidate;
remaining content is restored to its original name when vacant, otherwise retained under
`.jpdev-clean-removing-<token>` for operator inspection. A refusal can leave a partially removed
candidate; the remaining evidence is retained and requires assessment before another seal.
These checks address ordinary concurrent activity; they do not isolate a malicious process with
the same user identity, which can edit the tool or its ownership records.

Cleanup prints JSON Lines: candidate records contain `root`, root-relative `path`, logical `bytes`,
`reason` and `outcome`; `--yes` adds per-path `removed` or `refused` outcomes. The final record gives
`candidates`, logical `bytes`, `removed`, `refused` and `mode`. Totals count inspected candidates,
including candidates later refused during removal; they do not estimate physical reclaimed space.
No candidate or root failure is silently skipped. Exit 0 means no refusal (including a no-op or
absent root), 1 means cleanup reported a refusal/removal failure, and 2 means invalid arguments or
init/seal refusal. Init and seal print a short confirmation, rather than cleanup JSON Lines.

On-disk metadata is version 1 JSON in `DIR/.jpdev-clean`: `owner.json` records session token and
root device/inode; each seal is named by SHA-256 of its relative path and records version, session,
path, kind, candidate device/inode, seal time, content inventory, and optional base/removed paths.
Inventory keys are relative paths (empty for a standalone transcript file); values hold device,
inode, mode, logical size, nanosecond mtime/ctime, and SHA-256 for regular files. Records must be
regular, session-user-owned files, not writable by group/other, and at most 16 MiB. Root ownership
records and reserves are retained. They are local session evidence, never repository content.
Loading
Loading