diff --git a/AGENTS.md b/AGENTS.md index 5af7bab..56600c7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,6 +11,11 @@ Edit `AGENTS.md` and run `scripts/sync-agent-docs.sh` to regenerate `CLAUDE.md`. - If a check fails: read its first lines in the block, fix the cause, re-run that one check with `--only NAME`, then run the whole list once more. Do not loop on the whole list. - Start no CI watch. Push, report once, and stop; CI owns everything deterministic. +- Push with `jpdev push [BRANCH]`, or where the kit is not installed with + `git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`. + The machine's stored git credential can be another account's, and a plain `git push` is then + refused; the helper form uses the `gh` login. Never store a credential, never change global git + configuration, never push with force (`jpdev push` refuses `--force` and `-f`). ## Rules diff --git a/CLAUDE.md b/CLAUDE.md index 5af7bab..56600c7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,6 +11,11 @@ Edit `AGENTS.md` and run `scripts/sync-agent-docs.sh` to regenerate `CLAUDE.md`. - If a check fails: read its first lines in the block, fix the cause, re-run that one check with `--only NAME`, then run the whole list once more. Do not loop on the whole list. - Start no CI watch. Push, report once, and stop; CI owns everything deterministic. +- Push with `jpdev push [BRANCH]`, or where the kit is not installed with + `git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`. + The machine's stored git credential can be another account's, and a plain `git push` is then + refused; the helper form uses the `gh` login. Never store a credential, never change global git + configuration, never push with force (`jpdev push` refuses `--force` and `-f`). ## Rules diff --git a/README.md b/README.md index 0d9b998..01617a7 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,9 @@ the same name overrides it. `config/env.example` documents each setting with its | `JPDEV_NVM_DIR` | `$NVM_DIR` or `~/.nvm` | where node versions live | | `JPDEV_GO_SDK_DIR` | `~/sdk` | where a Go release the manifest names is looked for, when the go on PATH is another (`sdk/go1.26.5/bin/go`) | | `JPDEV_NAME_PATTERNS` | `~/.config/jpdev/name-patterns` | regexes the record gate refuses | +| `JPDEV_CODEX_MODEL_DEFAULT` | none (the model is always passed) | the model `jpdev codex run` uses when `--model` is not given | +| `JPDEV_CODEX_SANDBOX` | `workspace-write` | the sandbox mode of `jpdev codex run` | +| `JPDEV_CODEX_WORDS` | `~/.config/jpdev/codex-words.txt` | extra terms for `jpdev codex check-brief` | ## Commands @@ -96,6 +99,38 @@ the same name overrides it. `config/env.example` documents each setting with its remains, a line matches the name patterns, or the name-pattern file is missing or holds a pattern `grep -E` cannot compile, and posts with `gh ... comment --body-file`. A post that should go without the name check says `--no-name-check`. +- `jpdev push [BRANCH]`: pushes BRANCH (default: the current branch) to `origin` with exactly + `git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`, + so the push uses the `gh` login and no stored credential. It refuses `--force`, `-f` and a + `+` refspec. +- `jpdev codex run --brief FILE --model M --effort low|medium|high [--dir CLONE] [--repo OWNER/NAME] + [--base main] [--add PATH]... [--no-commit] [--no-push] [--no-pr] [--sandbox + workspace-write|danger-full-access] [--log FILE]`: checks the brief, then runs `codex exec` in + CLONE (a clone on a work branch that holds an `AGENTS.md`) with `-m`, + `-c model_reasoning_effort=`, `--sandbox` (network access on for `workspace-write`), stdin from + `/dev/null`, and a prompt of one line naming the repository's `AGENTS.md` as the rules followed + by the brief's text. The transcript goes to the log (default under `JPDEV_SCRATCH/logs`); the + command prints the final message and the exit code. The model is always passed: `--model`, or + `JPDEV_CODEX_MODEL_DEFAULT` from the config file. + + Codex in `workspace-write` cannot write `.git`, so it does not commit. It writes `msg.txt` (the + commit message) and `pr.md` (the pull request body) in the clone, and after `codex exec` returns + the command does the git part: it stages the changes to tracked files (`git add -u`, plus the + paths named with `--add`; no other untracked file, and never `msg.txt` or `pr.md`), commits with + `-s` and the message of `msg.txt` with the `Co-Authored-By: Codex ` trailer + added when `msg.txt` lacks it, pushes as `jpdev push` does, and opens the pull request with + `gh pr create --repo OWNER/NAME --base BASE --head BRANCH`, the subject of `msg.txt` as the + title and `pr.md` as the body. `--no-push` stops after the commit, `--no-pr` after the push and + `--no-commit` runs Codex alone. It stops with exit 1 and commits nothing when Codex changed no + tracked file or wrote no `msg.txt` (or no `pr.md`, when a pull request is to be opened), and + refuses a clone on `main`. The exit status is Codex's own when it fails, else 0 when every step + asked for was done. `--sandbox danger-full-access` is an explicit option only. +- `jpdev codex brief-template`: prints `templates/codex-brief.md`, a brief skeleton that tells + Codex to write `msg.txt` and `pr.md` and not to commit. +- `jpdev codex check-brief FILE`: exits 1 and names the neutral phrasing for each word of + [`config/codex-words.txt`](config/codex-words.txt) (and of the file `JPDEV_CODEX_WORDS` names) + that the brief holds. `codex run` runs the same check first and does not start the run on a + refusal. - `jpdev env`: the effective settings and where each came from. ## The caps @@ -103,7 +138,7 @@ the same name overrides it. `config/env.example` documents each setting with its | Toolchain | Cap | | --- | --- | | `node` | `lib/rsscap.sh`: the process group's resident memory and a timeout; never `ulimit -v`, which breaks vite and vitest | -| `go` | `flock` on the machine lock, `ulimit -v`, `GOFLAGS=-mod=mod`, a per-clone `GOCACHE`, `timeout` | +| `go` | `flock` on the machine lock, `ulimit -v`, `GOFLAGS=-mod=readonly` (a step never rewrites `go.mod`), a per-clone `GOCACHE`, `timeout` | | `python` | `ulimit -v` and `timeout` | | `shell` | `timeout` only | @@ -113,9 +148,12 @@ script uses the environment's interpreter and packages. ## What is here -- `bin/jpdev`, `lib/`: the command, the caps, the manifest check. +- `bin/jpdev`, `lib/`: the command, the caps, the manifest check, the brief check. +- `config/env.example`, `config/codex-words.txt`: the settings by example, and the base word list of + `jpdev codex check-brief`. - `templates/dev.json`, `templates/dev.schema.json`: the manifest by example and its JSON Schema. - `templates/check.sh`: the public runner a repository vendors as `scripts/dev/check.sh`. +- `templates/codex-brief.md`: the brief skeleton `jpdev codex brief-template` prints. - `templates/AGENTS.md`, `templates/CLAUDE.md`: the agent instructions (one text, generated from the other by `scripts/sync-agent-docs.sh`). - `templates/ci-check.yml`: the CI step list. diff --git a/bin/jpdev b/bin/jpdev index e3649e6..4e370df 100755 --- a/bin/jpdev +++ b/bin/jpdev @@ -20,6 +20,20 @@ usage: jpdev COMMAND [ARGS] record post (--pr N | --issue N) --file FILE [--repo OWNER/NAME] [--dry-run] [--no-name-check] reduce local paths, refuse a path, a name pattern, a missing or invalid name-pattern file, then post the file + push [BRANCH] push BRANCH (default: the current branch) to origin with the gh credential + helper and no stored credential; refuses --force and -f + codex run --brief FILE --model M --effort low|medium|high [--dir CLONE] [--repo OWNER/NAME] + [--base main] [--add PATH]... [--no-commit] [--no-push] [--no-pr] + [--sandbox workspace-write|danger-full-access] [--log FILE] + check the brief, run `codex exec` on it in CLONE (a work branch) with the + model, effort and sandbox explicit, stdin closed and the transcript in the + log; print the final message and the exit code; then commit the tracked + changes (msg.txt, signed off), push, and open the pull request (pr.md) + codex brief-template + print the brief skeleton: what Codex writes (msg.txt, pr.md) and does not do + codex check-brief FILE + refuse a brief that holds a word of the filter list, and suggest the + neutral phrasing env print the effective settings and where each came from --version print the version USAGE @@ -254,6 +268,205 @@ cmd_record() { gh "$kind" comment "$number" ${repo:+--repo "$repo"} --body-file "$reduced" } +# cmd_push [BRANCH]: the push form with the gh credential helper, never a stored credential. +cmd_push() { + local branch="" arg + for arg in "$@"; do + case $arg in + --force* | -f* | +*) jpdev_die "push: refused: a force push is never made (jpdev push takes a branch name only)" ;; + -*) jpdev_die "push: unknown argument $arg (jpdev push takes a branch name only)" ;; + *) [ -z "$branch" ] || jpdev_die "push: one branch only"; branch=$arg ;; + esac + done + repo_root > /dev/null + if [ -z "$branch" ]; then + branch=$(git symbolic-ref --short -q HEAD) || jpdev_die "push: HEAD is not on a branch; name the branch" + fi + git check-ref-format --branch "$branch" > /dev/null 2>&1 || jpdev_die "push: '$branch' is not a branch name" + jpdev_git_net push -u origin "$branch" +} + +# cmd_codex run|check-brief ... +cmd_codex() { + local sub=${1:-} + [ $# -eq 0 ] || shift + case $sub in + run) codex_run "$@" ;; + check-brief) codex_check_brief "$@" ;; + brief-template) codex_brief_template ;; + -h | --help | help) usage; exit 0 ;; + *) jpdev_die "codex: the subcommands are run, check-brief and brief-template" ;; + esac +} + +# codex_check_brief FILE: exit 1 and the neutral phrasing for each word of the filter list in FILE. +codex_check_brief() { + local file="" + while [ $# -gt 0 ]; do + case $1 in + -h | --help) usage; exit 0 ;; + -*) jpdev_die "codex check-brief: unknown argument $1" ;; + *) [ -z "$file" ] || jpdev_die "codex check-brief: one brief only"; file=$1; shift ;; + esac + done + [ -f "$file" ] || jpdev_die "codex check-brief: give the brief file" + local out status + out=$(python3 -I "$kit/lib/codex_words.py" "$file" "$kit/config/codex-words.txt" "$JPDEV_CODEX_WORDS") + status=$? + case $status in + 0) echo "jpdev codex check-brief: no word of the filter list in $file" ;; + 1) + printf 'jpdev codex check-brief: refused: %s holds words the content filter trips on:\n' "$file" >&2 + printf ' %s\n' "${out//$'\n'/$'\n' }" >&2 + echo "describe the documented refusal instead (the input, and the exact refusal the code must produce)" >&2 + exit 1 + ;; + *) jpdev_die "codex check-brief: the check failed (exit $status)" ;; + esac +} + +# codex_run: the Codex CLI build from a brief, then the git part (commit, push, pull request). +# Codex runs in workspace-write and cannot write .git, so the wrapper commits, pushes and opens +# the pull request itself from the msg.txt and pr.md that Codex wrote. +codex_run() { + local brief="" model=$JPDEV_CODEX_MODEL_DEFAULT effort="" dir=. sandbox=$JPDEV_CODEX_SANDBOX log="" + local repo="" base=main commit=1 push=1 pr=1 adds=() + while [ $# -gt 0 ]; do + case $1 in + --brief) need_value "$@"; brief=$2; shift 2 ;; + --model) need_value "$@"; model=$2; shift 2 ;; + --effort) need_value "$@"; effort=$2; shift 2 ;; + --dir) need_value "$@"; dir=$2; shift 2 ;; + --sandbox) need_value "$@"; sandbox=$2; shift 2 ;; + --log) need_value "$@"; log=$2; shift 2 ;; + --repo) need_value "$@"; repo=$2; shift 2 ;; + --base) need_value "$@"; base=$2; shift 2 ;; + --add) need_value "$@"; adds+=("$2"); shift 2 ;; + --no-commit) commit=0; shift ;; + --no-push) push=0; shift ;; + --no-pr) pr=0; shift ;; + -h | --help) usage; exit 0 ;; + *) jpdev_die "codex run: unknown argument $1" ;; + esac + done + [ -f "$brief" ] || jpdev_die "codex run: --brief must name a file" + [ -n "$model" ] || jpdev_die "codex run: --model is needed (the model is always passed; JPDEV_CODEX_MODEL_DEFAULT can name one)" + case $effort in + low | medium | high) ;; + *) jpdev_die "codex run: --effort is low, medium or high" ;; + esac + case $sandbox in + workspace-write | danger-full-access) ;; + *) jpdev_die "codex run: --sandbox is workspace-write or danger-full-access" ;; + esac + [ -d "$dir" ] || jpdev_die "codex run: --dir is not a directory" + dir=$(cd "$dir" && pwd) + [ -f "$dir/AGENTS.md" ] || jpdev_die "codex run: $dir holds no AGENTS.md, which the prompt names as the rules" + command -v codex > /dev/null || jpdev_die "codex run: codex is not on PATH" + local branch="" + if [ "$commit" -eq 1 ]; then + git -C "$dir" rev-parse --show-toplevel > /dev/null 2>&1 || jpdev_die "codex run: $dir is not a git clone (--no-commit runs codex alone)" + branch=$(git -C "$dir" symbolic-ref --short -q HEAD) || jpdev_die "codex run: HEAD is not on a branch" + case $branch in + main | master) jpdev_die "codex run: the clone is on $branch; make a work branch first (the wrapper commits and pushes the current branch)" ;; + esac + if [ "$push" -eq 1 ] && [ "$pr" -eq 1 ] && [ -z "$repo" ]; then + jpdev_die "codex run: --repo OWNER/NAME is needed to open the pull request (--no-pr stops after the push)" + fi + local path + for path in "${adds[@]}"; do + case $path in + msg.txt | pr.md | ./msg.txt | ./pr.md) jpdev_die "codex run: --add $path: msg.txt and pr.md are read, never committed" ;; + /* | .. | ../* | */../*) jpdev_die "codex run: --add takes a path inside the clone" ;; + esac + done + fi + brief=$(cd "$(dirname "$brief")" && pwd)/$(basename "$brief") + codex_check_brief "$brief" + if [ -z "$log" ]; then + mkdir -p "$JPDEV_SCRATCH/logs" + log=$JPDEV_SCRATCH/logs/codex-$(basename "$dir")-$(date +%Y%m%dT%H%M%S).log + fi + mkdir -p "$(dirname "$log")" + log=$(cd "$(dirname "$log")" && pwd)/$(basename "$log") + local final prompt + final=$(mktemp) + # a msg.txt or pr.md left by an earlier run is not this run's + rm -f -- "$dir/msg.txt" "$dir/pr.md" + prompt="The rules are this repository's AGENTS.md; follow them. +$(cat "$brief")" + local args=(exec -m "$model" -c "model_reasoning_effort=$effort" --sandbox "$sandbox") + if [ "$sandbox" = workspace-write ]; then args+=(-c sandbox_workspace_write.network_access=true); fi + args+=(-C "$dir" -o "$final" "$prompt") + jpdev_log "codex run: model $model, effort $effort, sandbox $sandbox, in $dir; transcript $log" + (cd "$dir" && codex "${args[@]}" < /dev/null > "$log" 2>&1) + local status=$? + echo "== codex final message ==" + if [ -s "$final" ]; then cat "$final"; else echo "(none written)"; fi + rm -f -- "$final" + echo "== codex exit $status (transcript: $log) ==" + if [ "$status" -ne 0 ] || [ "$commit" -eq 0 ]; then exit "$status"; fi + codex_publish "$dir" "$branch" "$push" "$pr" "$repo" "$base" "${adds[@]}" +} + +# codex_publish DIR BRANCH PUSH PR REPO BASE [ADD...]: stage the tracked changes (and the paths +# named with --add; never an untracked file otherwise, never msg.txt or pr.md), commit them signed +# off with the message in msg.txt plus the Codex trailer, push the branch, open the pull request +# from pr.md. Exits 0 when every step asked for was done, 1 when there was nothing to commit or a +# file was missing, and the failing command's status otherwise. +codex_publish() { + local dir=$1 branch=$2 push=$3 pr=$4 repo=$5 base=$6 + shift 6 + local trailer="Co-Authored-By: Codex " tmpd + cd "$dir" || exit 2 + if [ ! -s msg.txt ]; then + echo "jpdev codex run: refused: codex wrote no msg.txt (the commit message); nothing was committed" >&2 + exit 1 + fi + if [ "$pr" -eq 1 ] && [ "$push" -eq 1 ] && [ ! -s pr.md ]; then + echo "jpdev codex run: refused: codex wrote no pr.md (the pull request body); nothing was committed" >&2 + exit 1 + fi + git add -u || exit 1 + local path + for path in "$@"; do + git add -- "$path" || { echo "jpdev codex run: --add $path could not be staged" >&2; exit 1; } + done + if git diff --cached --name-only | grep -qxE 'msg\.txt|pr\.md'; then + git reset -q -- msg.txt pr.md + echo "jpdev codex run: refused: msg.txt or pr.md is tracked and changed; they are read, never committed" >&2 + exit 1 + fi + if git diff --cached --quiet; then + echo "jpdev codex run: nothing to commit: codex made no change to a tracked file" >&2 + exit 1 + fi + tmpd=$(mktemp -d) + git interpret-trailers --if-exists addIfDifferent --trailer "$trailer" msg.txt > "$tmpd/message" || { rm -rf -- "${tmpd:?}"; exit 1; } + git commit -q -s -F "$tmpd/message" || { local st=$?; rm -rf -- "${tmpd:?}"; echo "jpdev codex run: the commit failed" >&2; exit "$st"; } + local subject sha + subject=$(head -n 1 "$tmpd/message") + rm -rf -- "${tmpd:?}" + sha=$(git rev-parse --short HEAD) + echo "== jpdev codex run: committed $sha on $branch ==" + git diff-tree --no-commit-id --name-status -r HEAD + local stray + stray=$(git ls-files --others --exclude-standard | grep -vxE 'msg\.txt|pr\.md') + if [ -n "$stray" ]; then + echo "untracked files left out of the commit (pass --add PATH to include one):" + printf ' %s\n' "$stray" + fi + [ "$push" -eq 1 ] || exit 0 + (cmd_push "$branch") || { local st=$?; echo "jpdev codex run: the push failed; the commit $sha is local" >&2; exit "$st"; } + echo "== jpdev codex run: pushed $branch ==" + [ "$pr" -eq 1 ] || exit 0 + gh pr create --repo "$repo" --base "$base" --head "$branch" --title "$subject" --body-file pr.md || { local st=$?; echo "jpdev codex run: opening the pull request failed; the branch is pushed" >&2; exit "$st"; } + exit 0 +} + +# codex_brief_template prints the brief skeleton a Codex build starts from. +codex_brief_template() { cat "$kit/templates/codex-brief.md"; } + cmd_env() { local v cfg cfg=$(jpdev_config_file) @@ -274,6 +487,8 @@ case ${1:-} in check) shift; cmd_check "$@" ;; mutants) shift; cmd_mutants "$@" ;; record) shift; cmd_record "$@" ;; + push) shift; cmd_push "$@" ;; + codex) shift; cmd_codex "$@" ;; env) shift; cmd_env "$@" ;; --version | version) echo "jpdev $JPDEV_VERSION" ;; -h | --help | help | "") usage; [ -n "${1:-}" ] || exit 2 ;; diff --git a/config/codex-words.txt b/config/codex-words.txt new file mode 100644 index 0000000..b5b57ba --- /dev/null +++ b/config/codex-words.txt @@ -0,0 +1,39 @@ +# The kit's base list for `jpdev codex check-brief`: words that name attacks on systems or +# people, as verbs or nouns, which trip the content filter of the Codex service. +# One entry per line: TERM or TERM => the neutral phrasing to use instead. +# A TERM is matched case-insensitively at the start of a word, so "attack" also matches +# "attacker" and "attacking"; a TERM of several words matches across any run of spaces. +# Lines starting with # and blank lines are ignored. +# A person extends the list in the file named by JPDEV_CODEX_WORDS (config/env.example). +# +# Describe the documented refusal instead: the input the code receives, and the exact +# refusal (error, status or message) it must produce. + +attack => describe the input the code receives and the refusal it must produce +exploit => describe the malformed or unexpected input and the documented refusal +hack => describe the unsupported use and the refusal that documents it +breach => describe the boundary and the check that must hold at it +malicious => describe the input as unexpected, unsupported or not allowed +hostile => describe the input as unexpected, unsupported or not allowed +adversar => describe the input as unexpected or not allowed, and the refusal +jailbreak => describe the instruction the model must not follow and the refusal +prompt injection => describe the untrusted text in the input and the refusal that keeps it from acting as an instruction +code injection => describe the untrusted text in the input and the refusal +sql injection => describe the untrusted text in the input and the refusal +malware => describe the file or input the code must refuse +ransomware => describe the file or input the code must refuse +trojan => describe the file or input the code must refuse +backdoor => describe the access path that must not exist and the check that closes it +phish => describe the message or link the code must refuse +spoof => describe the identity claim that the check must reject +exfiltrat => describe the data that must not leave and the check that stops it +sabotage => describe the state that must not change and the check that holds it +poison => describe the input that must be rejected before it is stored +weaponiz => describe the use the code must refuse +privilege escalation => describe the permission that must not be granted and the check +denial of service => describe the limit the code enforces and the refusal past it +brute force => describe the limit on repeated attempts and the refusal past it +intrud => describe the access that must be refused +harass => describe the content the code must refuse +stalk => describe the content the code must refuse +doxx => describe the personal data that must not be published diff --git a/config/env.example b/config/env.example index c6553bb..8af1210 100644 --- a/config/env.example +++ b/config/env.example @@ -16,6 +16,9 @@ # Go steps: ulimit -v (KB) and the default timeout (s). The per-clone GOCACHE sits under JPDEV_SCRATCH. #JPDEV_GO_ULIMIT_KB=12000000 #JPDEV_GO_TIMEOUT_S=3000 +# Go steps always run with GOFLAGS=-mod=readonly and the per-clone GOCACHE, which lib/wrap.sh sets +# for the step: a GOFLAGS exported here or in the environment does not reach a Go step, and a step +# never rewrites go.mod or go.sum (a module the files lack fails the step, as it does in CI). # Python steps: ulimit -v (KB). #JPDEV_PY_ULIMIT_KB=4000000 @@ -33,6 +36,14 @@ # and refuses a file holding a pattern grep -E cannot compile. #JPDEV_NAME_PATTERNS="${XDG_CONFIG_HOME:-$HOME/.config}/jpdev/name-patterns" +# `jpdev codex run`: the model to pass when --model is not given (none by default: the model is +# always passed), the sandbox mode (workspace-write or danger-full-access) and a file of extra +# words for `jpdev codex check-brief`, in the format of config/codex-words.txt (a term, or +# "term => neutral phrasing", per line; the file may be absent). +#JPDEV_CODEX_MODEL_DEFAULT= +#JPDEV_CODEX_SANDBOX=workspace-write +#JPDEV_CODEX_WORDS="${XDG_CONFIG_HOME:-$HOME/.config}/jpdev/codex-words.txt" + # A manifest may name an environment variable a step needs (requires_env). The agent # repository's parity step needs the Desk checkout it compares against; export it here. #export JPDEV_DESK=/path/to/a/checkout/of/judgment-pack-desk diff --git a/docs/adopting.md b/docs/adopting.md index 90bc923..e109868 100644 --- a/docs/adopting.md +++ b/docs/adopting.md @@ -54,6 +54,15 @@ Copy `templates/AGENTS.md` to the repository root and generate `CLAUDE.md` from without writing, for a CI step. Add the repository's own rules below the shared ones in `AGENTS.md`, not in `CLAUDE.md`. +The shared rules state how to push: with the `gh` credential helper and never with force, because +the machine's stored git credential can be another account's and a plain `git push` is then +refused. Where the kit is installed, `jpdev push [BRANCH]` runs exactly that form +(`git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`) +and refuses `--force` and `-f`; the instructions give the full command for where it is not. The +kit changes no global git configuration. A repository that adopted an earlier copy of the +instructions copies `templates/AGENTS.md` again (keeping its own rules below the shared ones) and +regenerates `CLAUDE.md`. + ## 4. Add the CI workflow Copy `templates/ci-check.yml` to `.github/workflows/dev-check.yml`. It reads the toolchain diff --git a/docs/policy.md b/docs/policy.md index ee76b49..cd15ff6 100644 --- a/docs/policy.md +++ b/docs/policy.md @@ -18,7 +18,26 @@ are the ones a brief does not repeat. suites CI runs. It reads the diff, builds the change once, runs the probes it writes to test a claim, and breaks each safeguard the change adds (`jpdev mutants`) to see that a test notices. A safeguard whose test passes with the safeguard removed is a finding. -- **Each commit is signed off, no history is rewritten.** Pushes are never forced. +- **Each commit is signed off, no history is rewritten.** Pushes are never forced, and go + through the `gh` credential helper (`jpdev push`), never a stored credential. + +## Who builds and who reviews + +- **Codex builds non-boundary issues** from accepted designs and briefs, and Claude reviews + them. +- **Claude builds boundary decisions** (the architecture decision records that set a security + boundary), and Codex reviews them when its quota allows. +- A Codex build runs from a brief with `jpdev codex run`: the model and the effort are always + passed, the sandbox is `workspace-write` with network access, stdin is closed, and the prompt + names the repository's `AGENTS.md` as the rules. Codex cannot write `.git` in that sandbox, so + it does not commit: it writes `msg.txt` and `pr.md` (the brief skeleton from + `jpdev codex brief-template` says so), and `jpdev codex run` commits the tracked changes with + a sign-off, pushes through the `gh` credential helper and opens the pull request from those two + files. `danger-full-access` is chosen explicitly or not at all. +- A brief describes the documented refusal the code must produce (the input, and the exact + refusal), not the attack the refusal answers: the content filter of the Codex service can + refuse a brief that names attacks as verbs or nouns. `jpdev codex check-brief` refuses such a + brief before the run, and `jpdev codex run` runs the same check first. ## Reviews diff --git a/lib/codex_words.py b/lib/codex_words.py new file mode 100644 index 0000000..ce11742 --- /dev/null +++ b/lib/codex_words.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python3 +"""codex_words.py BRIEF WORDFILE...: list the words of the word files that the brief contains. + +A word file holds one entry per line: TERM or TERM => the neutral phrasing to use instead. +Lines that start with # and blank lines are ignored. A TERM matches case-insensitively at the +start of a word, with any letters after it, and a TERM of several words matches across any run +of whitespace and hyphens (so a term wrapped over two lines, or written with hyphens, is found; +the line reported is the line the match starts on). Evasion inside a single word (a letter +replaced or inserted) is out of scope: this is a vocabulary check, not a filter. A word file that does not exist is skipped (the config may extend the list). +Prints one line per hit and exits 1 when there is any, 0 when the brief is clean, 2 on a +usage or read error. +""" +import os +import re +import sys + +GENERIC = "describe the input and the documented refusal instead" + + +def load(path): + entries = [] + with open(path, encoding="utf-8") as handle: + for raw in handle: + line = raw.strip() + if not line or line.startswith("#"): + continue + term, _, suggestion = line.partition("=>") + term = " ".join(term.split()) + if term: + entries.append((term, suggestion.strip() or GENERIC)) + return entries + + +def main(argv): + if len(argv) < 3: + print("usage: codex_words.py BRIEF WORDFILE...", file=sys.stderr) + return 2 + try: + with open(argv[1], encoding="utf-8", errors="replace") as handle: + lines = handle.read().splitlines() + entries = [] + for path in argv[2:]: + if os.path.isfile(path): + entries.extend(load(path)) + except OSError as err: + print("codex_words.py: %s" % err, file=sys.stderr) + return 2 + text = "\n".join(lines) + hits = 0 + for term, suggestion in entries: + words = [re.escape(part) for part in term.split()] + pattern = re.compile(r"(?&2; } jpdev_die() { printf 'jpdev: %s\n' "$*" >&2; exit 2; } @@ -28,6 +29,9 @@ jpdev_default() { JPDEV_NVM_DIR) printf '%s' "${NVM_DIR:-$HOME/.nvm}" ;; JPDEV_GO_SDK_DIR) printf '%s/sdk' "$HOME" ;; JPDEV_NAME_PATTERNS) printf '%s/jpdev/name-patterns' "${XDG_CONFIG_HOME:-$HOME/.config}" ;; + JPDEV_CODEX_MODEL_DEFAULT) ;; # none: `jpdev codex run` always passes the model + JPDEV_CODEX_SANDBOX) printf workspace-write ;; + JPDEV_CODEX_WORDS) printf '%s/jpdev/codex-words.txt' "${XDG_CONFIG_HOME:-$HOME/.config}" ;; esac } diff --git a/lib/wrap.sh b/lib/wrap.sh index 981d0d6..fd919eb 100755 --- a/lib/wrap.sh +++ b/lib/wrap.sh @@ -2,7 +2,9 @@ # wrap.sh TOOLCHAIN -- command... # Runs one command under the cap its toolchain needs on a shared machine: # node the resident-memory cap (lib/rsscap.sh); never ulimit -v, which breaks vite and vitest -# go the machine lock, ulimit -v, GOFLAGS=-mod=mod, a per-clone GOCACHE, timeout +# go the machine lock, ulimit -v, GOFLAGS=-mod=readonly, a per-clone GOCACHE, timeout +# (-mod=readonly: a step never rewrites go.mod or go.sum, as in CI; a module the +# files lack fails the step instead of being added) # python ulimit -v and timeout # shell timeout only # JPDEV_STEP_TIMEOUT (seconds) replaces the toolchain's default timeout for this one command. @@ -34,7 +36,7 @@ case $toolchain in if ! flock -n "$JPDEV_LOCK" true; then jpdev_log "waiting for the machine lock"; fi # shellcheck disable=SC2016 # the script is for the inner bash, which expands it exec flock "$JPDEV_LOCK" bash -c ' - ulimit -v "$1"; export GOFLAGS=-mod=mod GOCACHE="$2"; tmo=$3; shift 3 + ulimit -v "$1"; export GOFLAGS=-mod=readonly GOCACHE="$2"; tmo=$3; shift 3 exec timeout -k 10 "$tmo" "$@"' jpdev-go "$JPDEV_GO_ULIMIT_KB" "$cache" "${step_timeout:-$JPDEV_GO_TIMEOUT_S}" "$@" ;; python) diff --git a/templates/AGENTS.md b/templates/AGENTS.md index 8c88535..13537d5 100644 --- a/templates/AGENTS.md +++ b/templates/AGENTS.md @@ -11,6 +11,11 @@ Edit `AGENTS.md` and run `scripts/sync-agent-docs.sh` to regenerate `CLAUDE.md`. - If a check fails: read its first lines in the block, fix the cause, re-run that one check with `--only NAME`, then run the whole list once more. Do not loop on the whole list. - Start no CI watch. Push, report once, and stop; CI owns everything deterministic. +- Push with `jpdev push [BRANCH]`, or where the kit is not installed with + `git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`. + The machine's stored git credential can be another account's, and a plain `git push` is then + refused; the helper form uses the `gh` login. Never store a credential, never change global git + configuration, never push with force (`jpdev push` refuses `--force` and `-f`). ## Rules diff --git a/templates/CLAUDE.md b/templates/CLAUDE.md index 8c88535..13537d5 100644 --- a/templates/CLAUDE.md +++ b/templates/CLAUDE.md @@ -11,6 +11,11 @@ Edit `AGENTS.md` and run `scripts/sync-agent-docs.sh` to regenerate `CLAUDE.md`. - If a check fails: read its first lines in the block, fix the cause, re-run that one check with `--only NAME`, then run the whole list once more. Do not loop on the whole list. - Start no CI watch. Push, report once, and stop; CI owns everything deterministic. +- Push with `jpdev push [BRANCH]`, or where the kit is not installed with + `git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH`. + The machine's stored git credential can be another account's, and a plain `git push` is then + refused; the helper form uses the `gh` login. Never store a credential, never change global git + configuration, never push with force (`jpdev push` refuses `--force` and `-f`). ## Rules diff --git a/templates/codex-brief.md b/templates/codex-brief.md new file mode 100644 index 0000000..362adb7 --- /dev/null +++ b/templates/codex-brief.md @@ -0,0 +1,45 @@ +# Brief: TITLE (issue #N of OWNER/NAME) + +## Rules +Read this repository's AGENTS.md and follow it. Work only in the current directory. Start no +sub-agents. Never use a pattern kill (`pkill`, `killall`). + +You cannot write `.git` here, so do not run `git commit`, `git push` or `gh`. Edit the files, run +the repository's check list once (`scripts/dev/check.sh`, or `jpdev check`), and write the two +files below. The wrapper that started you (`jpdev codex run`) then commits the changes to tracked +files with a sign-off, pushes the branch and opens the pull request. + +## Goal +What the issue asks for, in one or two sentences. Link the accepted design. + +## What to build +1. The first change, with the file it belongs in. +2. The next change. + +Describe each refusal the code must produce as the input it receives and the exact error, status +or message it returns. + +## Tests +A test for every new behaviour. Write vectors by hand. Break the change and watch the test fail +before you rely on it. + +## Files you write +Write both in the repository root. They are read, never committed. + +- `msg.txt`: the commit message. A subject of at most 72 characters in the imperative, a blank + line, then a plain, factual body (what changed and why). Leave out `Signed-off-by` and + `Co-Authored-By` lines: the wrapper adds them. +- `pr.md`: the pull request body. Its first line is `Closes #N.`; then the sections + `## What it adds`, `## Checks` (paste the summary block that the check list printed) and + `## Not in this PR`; then a last line saying a Codex session wrote the change. + +No machine-local path and no personal name in any file, `msg.txt` or `pr.md`. + +## New files +The wrapper commits changes to files that are already tracked. If you create a file, name it in +your final message (one path per line under the heading `New files`) so that it can be passed +with `--add`. + +## Report +Your final message is under 10 lines: what changed, the check summary block, the new files, and +anything you left out and why. diff --git a/tests/run.sh b/tests/run.sh index 7b97e79..27d2b10 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -7,7 +7,7 @@ work=$(mktemp -d) trap 'rm -rf -- "${work:?}"' EXIT export JPDEV_SCRATCH=$work/scratch XDG_CONFIG_HOME=$work/config HOME=$work/home mkdir -p "$HOME" "$XDG_CONFIG_HOME" -unset CI JPDEV_VENV JPDEV_LOCK JPDEV_NODE_RSS_MB JPDEV_NODE_TIMEOUT_S JPDEV_GO_ULIMIT_KB JPDEV_GO_TIMEOUT_S JPDEV_PY_ULIMIT_KB JPDEV_NVM_DIR JPDEV_GO_SDK_DIR JPDEV_NAME_PATTERNS JPDEV_FIXTURE_UNSET +unset CI JPDEV_VENV JPDEV_LOCK JPDEV_NODE_RSS_MB JPDEV_NODE_TIMEOUT_S JPDEV_GO_ULIMIT_KB JPDEV_GO_TIMEOUT_S JPDEV_PY_ULIMIT_KB JPDEV_NVM_DIR JPDEV_GO_SDK_DIR JPDEV_NAME_PATTERNS JPDEV_CODEX_MODEL_DEFAULT JPDEV_CODEX_SANDBOX JPDEV_CODEX_WORDS JPDEV_FIXTURE_UNSET # What jpdev check exports to its steps: a run of these tests under it must not inherit it. unset JPDEV_ROOT JPDEV_WRAP JPDEV_NODE_BIN JPDEV_GO_BIN JPDEV_LOGDIR JPDEV_STEP_TIMEOUT jpdev=$kit/bin/jpdev @@ -89,9 +89,41 @@ expect "jpdev check applies the Go address-space cap (a 1 MB cap fails the step) out=$("$jpdev" check --manifest "$fixture" --only go-env 2>/dev/null) && s=0 || s=$? expect "jpdev check runs a Go step" 0 'PASS +go-env' "$out" "$s" golog=$(cat "$JPDEV_SCRATCH"/logs/*/go-env.log 2>/dev/null) -expect "a Go step runs with GOFLAGS=-mod=mod" 0 '^-mod=mod$' "$golog" 0 +expect "a Go step runs with GOFLAGS=-mod=readonly" 0 '^-mod=readonly$' "$golog" 0 +absent "a Go step does not run with -mod=mod" '^-mod=mod$' "$golog" +out=$(GOFLAGS=-mod=mod "$jpdev" check --manifest "$fixture" --only go-env 2>/dev/null) && s=0 || s=$? +golog=$(cat "$JPDEV_SCRATCH"/logs/*/go-env.log 2>/dev/null | tail -n 2) +expect "a GOFLAGS in the environment does not reach a Go step" 0 '^-mod=readonly$' "$golog" 0 expect "a Go step runs with a per-clone cache under the scratch directory" 0 "^$JPDEV_SCRATCH/gocache/" "$golog" 0 +# a Go step leaves go.mod byte-identical: under -mod=mod, go vet drops the "// indirect" marker of a +# module the code imports directly; under -mod=readonly it does not (the module is a local replace) +gomod=$work/gomod +mkdir -p "$gomod/dep" +git init -q -b main "$gomod" +printf 'module example.com/dep\n\ngo 1.21\n' > "$gomod/dep/go.mod" +printf 'package dep\n\nfunc F() int { return 1 }\n' > "$gomod/dep/dep.go" +printf 'module example.com/app\n\ngo 1.21\n\nrequire example.com/dep v0.0.0 // indirect\n\nreplace example.com/dep => ./dep\n' > "$gomod/go.mod" +printf 'package main\n\nimport "example.com/dep"\n\nfunc main() { _ = dep.F() }\n' > "$gomod/main.go" +cat > "$gomod/dev.json" <<'JSON' +{"version": 1, "toolchains": {"go": {}}, "checks": [{"name": "vet", "run": "go vet . && go build -o /dev/null .", "toolchain": "go"}]} +JSON +before=$(sha256sum "$gomod/go.mod") +out=$(cd "$gomod" && "$jpdev" check 2>&1) && s=0 || s=$? +expect "a Go step that vets and builds passes on a read-only module" 0 'PASS +vet' "$out" "$s" +after=$(sha256sum "$gomod/go.mod") +if [ "$before" = "$after" ]; then ok "a Go step leaves go.mod byte-identical"; else bad "a Go step leaves go.mod byte-identical" "$(cat "$gomod/go.mod")"; fi +# the same step under -mod=mod does rewrite it, which is what the test above must catch +(cd "$gomod" && GOFLAGS=-mod=mod go vet . > /dev/null 2>&1) +if [ "$before" != "$(sha256sum "$gomod/go.mod")" ]; then ok "the test module is one that -mod=mod rewrites"; else bad "the test module is one that -mod=mod rewrites"; fi +printf 'module example.com/app\n\ngo 1.21\n\nrequire example.com/dep v0.0.0 // indirect\n\nreplace example.com/dep => ./dep\n' > "$gomod/go.mod" +# a module the files lack fails the step and leaves go.mod alone +printf 'package main\n\nimport _ "example.com/missing"\n\nfunc main() {}\n' > "$gomod/main.go" +before=$(sha256sum "$gomod/go.mod") +out=$(cd "$gomod" && GOPROXY=off "$jpdev" check 2>&1) && s=0 || s=$? +expect "a Go step that needs a module go.mod lacks fails" 1 'FAIL +vet' "$out" "$s" +if [ "$before" = "$(sha256sum "$gomod/go.mod")" ]; then ok "a failed Go step leaves go.mod byte-identical"; else bad "a failed Go step leaves go.mod byte-identical"; fi + # --- the node cap "$kit/lib/rsscap.sh" 6000 1 -- sleep 5 2>/dev/null; s=$? expect "rsscap stops a run that passes its time limit with 124" 124 "" "" "$s" @@ -319,6 +351,326 @@ expect "record refuses when the name check itself fails" 1 'the name check faile out=$("$jpdev" record post --file "$clean" --dry-run 2>&1) && s=0 || s=$? expect "record needs a pull request or an issue" 2 'give --pr N or --issue N' "$out" "$s" +# --- jpdev push +real_git=$(command -v git) +mkdir -p "$work/gitstub" +cat > "$work/gitstub/git" <> "\$STUB_GIT_ARGS.calls"; printf '%s\\0' "\$@" > "\$STUB_GIT_ARGS"; exit "\${STUB_GIT_EXIT:-0}"; fi +done +exec $real_git "\$@" +STUB +chmod +x "$work/gitstub/git" +pushorigin=$work/pushorigin.git +pushclone=$work/pushclone +git init -q --bare -b main "$pushorigin" +git init -q -b main "$pushclone" +git -C "$pushclone" -c user.name=test -c user.email=test@example.com commit -q --allow-empty -m seed +git -C "$pushclone" remote add origin "$pushorigin" +git -C "$pushclone" switch -q -c topic +export STUB_GIT_ARGS=$work/git-args +rm -f "$STUB_GIT_ARGS" +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push topic 2>&1) && s=0 || s=$? +expect "push with a branch name runs the push" 0 "" "$out" "$s" +want=$(printf '%s\0' -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin topic | od -An -c) +got=$(od -An -c < "$STUB_GIT_ARGS" 2>/dev/null) +if [ "$want" = "$got" ]; then ok "push runs exactly the gh credential-helper form"; else bad "push runs exactly the gh credential-helper form" "$(tr '\0' ' ' < "$STUB_GIT_ARGS")"; fi +rm -f "$STUB_GIT_ARGS" +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push 2>&1) && s=0 || s=$? +expect "push without a branch pushes the current branch" 0 "" "$out" "$s" +if [ "$(tr '\0' ' ' < "$STUB_GIT_ARGS")" = "-c credential.helper= -c credential.helper=!gh auth git-credential push -u origin topic " ]; then ok "push without a branch names the current branch"; else bad "push without a branch names the current branch" "$(tr '\0' ' ' < "$STUB_GIT_ARGS")"; fi +for flag in --force -f --force-with-lease --force-if-includes -fu +topic; do + rm -f "$STUB_GIT_ARGS" + out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push "$flag" topic 2>&1) && s=0 || s=$? + expect "push refuses $flag" 2 'refused: a force push is never made' "$out" "$s" + if [ -e "$STUB_GIT_ARGS" ]; then bad "push $flag reached git push"; else ok "push $flag did not reach git push"; fi +done +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push topic --force 2>&1) && s=0 || s=$? +expect "push refuses --force after the branch" 2 'refused: a force push is never made' "$out" "$s" +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push -u 2>&1) && s=0 || s=$? +expect "push refuses an option it does not know" 2 'unknown argument -u' "$out" "$s" +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push a b 2>&1) && s=0 || s=$? +expect "push takes one branch" 2 'one branch only' "$out" "$s" +out=$(cd "$pushclone" && PATH=$work/gitstub:$PATH "$jpdev" push 'a..b' 2>&1) && s=0 || s=$? +expect "push refuses a name that is not a branch name" 2 'not a branch name' "$out" "$s" +out=$(cd "$work" && PATH=$work/gitstub:$PATH "$jpdev" push topic 2>&1) && s=0 || s=$? +expect "push outside a clone fails" 2 'not inside a git clone' "$out" "$s" +out=$(cd "$pushclone" && STUB_GIT_EXIT=3 PATH=$work/gitstub:$PATH "$jpdev" push topic 2>&1) && s=0 || s=$? +expect "push returns the exit status of git push" 3 "" "$out" "$s" +out=$(cd "$pushclone" && "$jpdev" push topic 2>&1) && s=0 || s=$? +expect "push pushes to a repository and sets the upstream" 0 "" "$out" "$s" +if [ "$(git -C "$pushorigin" rev-parse refs/heads/topic 2>/dev/null)" = "$(git -C "$pushclone" rev-parse topic)" ] \ + && [ "$(git -C "$pushclone" config branch.topic.remote)" = origin ]; then ok "the branch is on origin with its upstream set"; else bad "the branch is on origin with its upstream set"; fi +unset STUB_GIT_ARGS +# the agent instructions and the docs state the push form +pushform="git -c credential.helper= -c 'credential.helper=!gh auth git-credential' push -u origin BRANCH" +for doc in templates/AGENTS.md templates/CLAUDE.md AGENTS.md CLAUDE.md; do + if grep -qF -- "$pushform" "$kit/$doc" && grep -qF 'jpdev push' "$kit/$doc"; then ok "$doc states the push form and names jpdev push"; else bad "$doc states the push form and names jpdev push"; fi +done +if grep -qF 'jpdev push' "$kit/docs/adopting.md"; then ok "docs/adopting.md mentions jpdev push"; else bad "docs/adopting.md mentions jpdev push"; fi + +# --- jpdev codex (a stub codex first on PATH: the real one is never called) +mkdir -p "$work/codexbin" "$work/codexclone" +cat > "$work/codexbin/codex" <<'STUB' +#!/usr/bin/env bash +printf '%s\0' "$@" > "$STUB_CODEX_ARGS" +pwd > "$STUB_CODEX_CWD" +timeout 2 cat > "$STUB_CODEX_STDIN" +while [ $# -gt 0 ]; do + case $1 in -o) printf 'stub final message\n' > "$2" ;; esac + shift +done +if [ -n "${STUB_CODEX_EDIT:-}" ]; then + echo "edited by codex" >> file.txt + echo "new file" > created.txt +fi +if [ -n "${STUB_CODEX_MSG:-}" ]; then + cp "$STUB_CODEX_MSG" msg.txt + if [ -z "${STUB_CODEX_NO_PR:-}" ]; then printf 'Closes #1.\n\n## What it adds\nA line.\n' > pr.md; fi +fi +echo "stub transcript line" +echo "stub stderr line" >&2 +exit "${STUB_CODEX_EXIT:-0}" +STUB +chmod +x "$work/codexbin/codex" +export STUB_CODEX_ARGS=$work/codex-args STUB_CODEX_CWD=$work/codex-cwd STUB_CODEX_STDIN=$work/codex-stdin +oldpath=$PATH +PATH=$work/codexbin:$PATH +printf '# Rules\n' > "$work/codexclone/AGENTS.md" +printf 'Add the flag.\nIt must refuse an unknown value with exit 2.\n' > "$work/brief.md" +codex_args() { rm -f "$STUB_CODEX_ARGS" "$STUB_CODEX_CWD" "$STUB_CODEX_STDIN"; } +has_arg() { tr '\0' '\n' < "$STUB_CODEX_ARGS" 2>/dev/null | grep -qxF -- "$1"; } +codex_args +out=$(echo junk | "$jpdev" codex run --no-commit --brief "$work/brief.md" --model model-x --effort high --dir "$work/codexclone" --log "$work/codex.log" 2>/dev/null) && s=0 || s=$? +expect "codex run prints the final message" 0 '^stub final message$' "$out" "$s" +expect "codex run prints the exit code" 0 '^== codex exit 0 ' "$out" "$s" +for a in exec -m model-x 'model_reasoning_effort=high' --sandbox workspace-write 'sandbox_workspace_write.network_access=true' -C "$work/codexclone"; do + if has_arg "$a"; then ok "codex run passes $a"; else bad "codex run passes $a" "$(tr '\0' ' ' < "$STUB_CODEX_ARGS")"; fi +done +if [ "$(tr '\0' '\n' < "$STUB_CODEX_ARGS" | sed -n '1,7p' | paste -sd' ')" = "exec -m model-x -c model_reasoning_effort=high --sandbox workspace-write" ]; then ok "codex run passes the model and effort as flags and values"; else bad "codex run passes the model and effort as flags and values" "$(tr '\0' ' ' < "$STUB_CODEX_ARGS")"; fi +prompt=$(python3 -I -c 'import sys; a = open(sys.argv[1], "rb").read().split(b"\0"); sys.stdout.write(a[-2].decode())' "$STUB_CODEX_ARGS") +if [ "$(head -n 1 <<< "$prompt")" = "The rules are this repository's AGENTS.md; follow them." ]; then ok "the prompt opens with one line naming AGENTS.md as the rules"; else bad "the prompt opens with one line naming AGENTS.md as the rules" "$prompt"; fi +if [ "$(tail -n +2 <<< "$prompt")" = "$(cat "$work/brief.md")" ]; then ok "the prompt is that line and the brief's text"; else bad "the prompt is that line and the brief's text" "$prompt"; fi +if [ "$(cat "$STUB_CODEX_CWD")" = "$(cd "$work/codexclone" && pwd -P)" ]; then ok "codex runs in the directory given"; else bad "codex runs in the directory given" "$(cat "$STUB_CODEX_CWD")"; fi +if [ -f "$STUB_CODEX_STDIN" ] && [ ! -s "$STUB_CODEX_STDIN" ]; then ok "codex's stdin is /dev/null, whatever jpdev reads"; else bad "codex's stdin is /dev/null, whatever jpdev reads" "$(cat "$STUB_CODEX_STDIN")"; fi +log=$(cat "$work/codex.log") +expect "the transcript (stdout) is in the log" 0 '^stub transcript line$' "$log" 0 +expect "the transcript (stderr) is in the log" 0 '^stub stderr line$' "$log" 0 +absent "the final message is not part of the transcript" 'stub final message' "$log" +out=$(STUB_CODEX_EXIT=7 "$jpdev" codex run --no-commit --brief "$work/brief.md" --model model-x --effort low --dir "$work/codexclone" --log "$work/codex7.log" 2>/dev/null) && s=0 || s=$? +expect "codex run returns codex's exit code" 7 '^== codex exit 7 ' "$out" "$s" +codex_args +out=$("$jpdev" codex run --no-commit --brief "$work/brief.md" --model model-x --effort medium --dir "$work/codexclone" --sandbox danger-full-access --log "$work/codex-d.log" 2>/dev/null) && s=0 || s=$? +expect "codex run accepts --sandbox danger-full-access" 0 'codex exit 0' "$out" "$s" +if has_arg danger-full-access && ! has_arg 'sandbox_workspace_write.network_access=true'; then ok "danger-full-access is passed without the workspace-write network setting"; else bad "danger-full-access is passed without the workspace-write network setting" "$(tr '\0' ' ' < "$STUB_CODEX_ARGS")"; fi +codex_args +out=$(cd "$work/codexclone" && JPDEV_CODEX_MODEL_DEFAULT=model-d JPDEV_CODEX_SANDBOX=danger-full-access "$jpdev" codex run --no-commit --brief "$work/brief.md" --effort low 2>/dev/null) && s=0 || s=$? +expect "codex run uses the configured default model, sandbox and the current directory" 0 'codex exit 0' "$out" "$s" +if has_arg model-d && has_arg danger-full-access; then ok "the defaults reach codex"; else bad "the defaults reach codex" "$(tr '\0' ' ' < "$STUB_CODEX_ARGS")"; fi +logline=$(grep -E '^== codex exit' <<< "$out") +if [ -f "$(sed -E 's/.*transcript: (.*)\) ==$/\1/' <<< "$logline")" ]; then ok "with no --log the transcript is under the scratch directory"; else bad "with no --log the transcript is under the scratch directory" "$logline"; fi +codex_args +out=$("$jpdev" codex run --brief "$work/brief.md" --effort high --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run refuses a run with no model" 2 '--model is needed' "$out" "$s" +out=$("$jpdev" codex run --brief "$work/brief.md" --model m --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run refuses a run with no effort" 2 'low, medium or high' "$out" "$s" +out=$("$jpdev" codex run --brief "$work/brief.md" --model m --effort max --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run refuses an effort it does not know" 2 'low, medium or high' "$out" "$s" +out=$("$jpdev" codex run --brief "$work/brief.md" --model m --effort low --sandbox read-only --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run refuses a sandbox it does not know" 2 'workspace-write or danger-full-access' "$out" "$s" +out=$("$jpdev" codex run --brief "$work/nobrief.md" --model m --effort low --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run needs the brief file" 2 '--brief must name a file' "$out" "$s" +mkdir -p "$work/norules" +out=$("$jpdev" codex run --brief "$work/brief.md" --model m --effort low --dir "$work/norules" 2>&1) && s=0 || s=$? +expect "codex run refuses a directory with no AGENTS.md" 2 'holds no AGENTS.md' "$out" "$s" +if [ -e "$STUB_CODEX_ARGS" ]; then bad "a refused run did not start codex"; else ok "a refused run did not start codex"; fi + +# check-brief: the base list, the neutral phrasing, the config extension +printf 'Reject a request with an unexpected value and return the documented error.\nA shackle is a ring.\n' > "$work/clean-brief.md" +out=$("$jpdev" codex check-brief "$work/clean-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief accepts a brief with no listed word" 0 'no word of the filter list' "$out" "$s" +printf 'Defend against an attacker who sends a prompt injection.\nIt must stop the EXPLOITATION of the parser.\n' > "$work/dirty-brief.md" +out=$("$jpdev" codex check-brief "$work/dirty-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief refuses a brief with a listed word" 1 'refused: .* holds words the content filter trips on' "$out" "$s" +expect "check-brief names the line and the word" 1 'line 1: "attacker"' "$out" "$s" +expect "check-brief matches a phrase" 1 'line 1: "prompt injection": describe the untrusted text' "$out" "$s" +expect "check-brief matches regardless of case and ending" 1 'line 2: "EXPLOITATION"' "$out" "$s" +expect "check-brief suggests the neutral phrasing" 1 'describe the input the code receives and the refusal it must produce' "$out" "$s" +expect "check-brief points at the documented refusal" 1 'describe the documented refusal instead' "$out" "$s" +absent "check-brief does not flag a word that merely contains a listed one" 'shackle' "$out" +n=0 +while IFS= read -r line; do + case $line in '' | '#'*) continue ;; esac + term=${line%%=>*}; term=${term%"${term##*[![:space:]]}"} + n=$((n + 1)) + printf 'The change handles %s here.\n' "$term" > "$work/term-brief.md" + out=$("$jpdev" codex check-brief "$work/term-brief.md" 2>&1) && s=0 || s=$? + if [ "$s" -ne 1 ]; then bad "check-brief refuses the base-list term '$term'" "$out"; fi + [ -z "${term_failed:-}" ] || true +done < "$kit/config/codex-words.txt" +if [ "$n" -ge 20 ]; then ok "check-brief refuses each of the $n terms of config/codex-words.txt"; else bad "the base word list holds at least 20 terms" "$n"; fi +printf 'Guard against a prompt\ninjection in the page,\nand a prompt-injection in a tool result, and a Prompt - Injection.\nA denial-of-service limit and a denial of\nservice limit.\n' > "$work/wrapped-brief.md" +out=$("$jpdev" codex check-brief "$work/wrapped-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief finds a term wrapped over two lines, reported at its first line" 1 'line 1: "prompt injection"' "$out" "$s" +expect "check-brief finds a hyphenated term" 1 'line 3: "prompt-injection"' "$out" "$s" +expect "check-brief finds a term with mixed spaces and hyphens" 1 'line 3: "Prompt - Injection"' "$out" "$s" +expect "check-brief finds a hyphenated three-word term" 1 'line 4: "denial-of-service"' "$out" "$s" +expect "check-brief finds a three-word term wrapped over two lines" 1 'line 4: "denial of service"' "$out" "$s" +printf 'The prompt and the injection of values are separate words.\n' > "$work/apart-brief.md" +out=$("$jpdev" codex check-brief "$work/apart-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief does not join words that are not adjacent" 0 'no word of the filter list' "$out" "$s" +printf 'extra-term => say it plainly\n' > "$work/extra-words.txt" +printf 'Handle the extra-term case.\n' > "$work/extra-brief.md" +out=$("$jpdev" codex check-brief "$work/extra-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief accepts a term the base list lacks" 0 'no word of the filter list' "$out" "$s" +out=$(JPDEV_CODEX_WORDS=$work/extra-words.txt "$jpdev" codex check-brief "$work/extra-brief.md" 2>&1) && s=0 || s=$? +expect "check-brief refuses a term the config adds, with its phrasing" 1 'line 1: "extra-term": say it plainly' "$out" "$s" +out=$(JPDEV_CODEX_WORDS=$work/extra-words.txt "$jpdev" codex check-brief "$work/dirty-brief.md" 2>&1) && s=0 || s=$? +expect "the config extends the base list and does not replace it" 1 'line 1: "attacker"' "$out" "$s" +out=$("$jpdev" codex check-brief "$work/nobrief.md" 2>&1) && s=0 || s=$? +expect "check-brief needs the file" 2 'give the brief file' "$out" "$s" +codex_args +out=$("$jpdev" codex run --no-commit --brief "$work/dirty-brief.md" --model m --effort low --dir "$work/codexclone" 2>&1) && s=0 || s=$? +expect "codex run refuses a brief with a listed word" 1 'refused: .* holds words the content filter trips on' "$out" "$s" +if [ -e "$STUB_CODEX_ARGS" ]; then bad "a refused brief did not start codex"; else ok "a refused brief did not start codex"; fi +out=$("$jpdev" codex 2>&1) && s=0 || s=$? +expect "codex needs a subcommand" 2 'the subcommands are run, check-brief and brief-template' "$out" "$s" +# the git part: commit, push, pull request, from the msg.txt and pr.md codex wrote +export STUB_GIT_ARGS=$work/git-args +mkdir -p "$work/ghbin" +cat > "$work/ghbin/gh" <<'STUB' +#!/usr/bin/env bash +echo x >> "$STUB_GH_CALLS" +printf '%s\0' "$@" > "$STUB_GH_ARGS" +while [ $# -gt 0 ]; do + case $1 in --body-file) cp "$2" "$STUB_GH_BODY" ;; esac + shift +done +echo "https://github.com/Example/repo/pull/1" +STUB +chmod +x "$work/ghbin/gh" +export STUB_GH_CALLS=$work/gh-calls STUB_GH_ARGS=$work/gh-args STUB_GH_BODY=$work/gh-body +export GIT_AUTHOR_NAME=Test GIT_AUTHOR_EMAIL=test@example.com GIT_COMMITTER_NAME=Test GIT_COMMITTER_EMAIL=test@example.com +cxorigin=$work/cxorigin.git +cxrepo=$work/cxrepo +git init -q --bare -b main "$cxorigin" +git init -q -b main "$cxrepo" +printf '# Rules\n' > "$cxrepo/AGENTS.md" +printf 'one\n' > "$cxrepo/file.txt" +git -C "$cxrepo" add AGENTS.md file.txt +git -C "$cxrepo" commit -q -m seed +git -C "$cxrepo" remote add origin "$cxorigin" +git -C "$cxrepo" push -q origin main +git -C "$cxrepo" switch -q -c work +printf 'Add the line\n\nExplain why.\n' > "$work/msg-plain.txt" +printf 'Add the line\n\nExplain why.\n\nCo-Authored-By: Codex \n' > "$work/msg-trailer.txt" +cx_reset() { + git -C "$cxrepo" reset -q --hard origin/main + git -C "$cxrepo" clean -q -fdx + git -C "$cxrepo" switch -q -C work + rm -f "$STUB_GIT_ARGS" "$STUB_GIT_ARGS.calls" "$STUB_GH_CALLS" "$STUB_GH_ARGS" "$STUB_GH_BODY" "$STUB_CODEX_ARGS" +} +cx_run() { # extra arguments to codex run + PATH=$work/gitstub:$work/ghbin:$PATH "$jpdev" codex run --brief "$work/brief.md" --model m --effort low --dir "$cxrepo" --repo Example/repo --log "$work/cx.log" "$@" 2>&1 +} +cx_calls() { if [ -f "$1" ]; then wc -l < "$1" | tr -d ' '; else echo 0; fi; } +base=$(git -C "$cxrepo" rev-parse HEAD) + +cx_reset +printf 'untracked and not named\n' > "$cxrepo/stray.txt" +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "codex run commits, pushes and opens the pull request" 0 'committed [0-9a-f]+ on work' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 1 ]; then ok "codex run makes one commit"; else bad "codex run makes one commit"; fi +if [ "$(git -C "$cxrepo" show --name-only --format= HEAD)" = file.txt ]; then ok "the commit holds the tracked change only"; else bad "the commit holds the tracked change only" "$(git -C "$cxrepo" show --name-only --format= HEAD)"; fi +expect "the untracked files left out are listed" 0 'stray.txt' "$out" "$s" +msg=$(git -C "$cxrepo" log -1 --format=%B) +expect "the commit has the subject of msg.txt" 0 '^Add the line$' "$msg" 0 +expect "the commit is signed off" 0 '^Signed-off-by: Test $' "$msg" 0 +expect "the commit has the Codex trailer, added" 0 '^Co-Authored-By: Codex $' "$msg" 0 +if [ "$(grep -c '^Co-Authored-By: Codex' <<< "$msg")" = 1 ]; then ok "the Codex trailer appears once"; else bad "the Codex trailer appears once" "$msg"; fi +if [ "$(cx_calls "$STUB_GIT_ARGS.calls")" = 1 ] && [ "$(tr '\0' ' ' < "$STUB_GIT_ARGS")" = "-c credential.helper= -c credential.helper=!gh auth git-credential push -u origin work " ]; then ok "one push, in the credential-helper form"; else bad "one push, in the credential-helper form" "$(tr '\0' ' ' < "$STUB_GIT_ARGS" 2>/dev/null)"; fi +if [ "$(cx_calls "$STUB_GH_CALLS")" = 1 ]; then ok "one gh pr create call"; else bad "one gh pr create call"; fi +if [ "$(tr '\0' ' ' < "$STUB_GH_ARGS")" = "pr create --repo Example/repo --base main --head work --title Add the line --body-file pr.md " ]; then ok "the pull request is made with --repo, --base, the subject and pr.md"; else bad "the pull request is made with --repo, --base, the subject and pr.md" "$(tr '\0' ' ' < "$STUB_GH_ARGS")"; fi +expect "the pull request body is pr.md" 0 '^Closes #1\.$' "$(cat "$STUB_GH_BODY")" 0 + +cx_reset +out=$(STUB_GIT_EXIT=3 STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "a failed push returns git's status and says the commit is local" 3 'the push failed; the commit [0-9a-f]+ is local' "$out" "$s" +if [ "$(cx_calls "$STUB_GH_CALLS")" = 0 ]; then ok "a failed push opens no pull request"; else bad "a failed push opens no pull request"; fi +cx_reset +out=$(STUB_GIT_FAIL_REF=1 STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "a push that dies before git push still says the commit is local" 2 'the push failed; the commit [0-9a-f]+ is local' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 1 ]; then ok "the commit is kept when the push fails"; else bad "the commit is kept when the push fails"; fi + +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-trailer.txt cx_run) && s=0 || s=$? +expect "codex run takes a msg.txt that has the trailer" 0 'committed' "$out" "$s" +msg=$(git -C "$cxrepo" log -1 --format=%B) +if [ "$(grep -c '^Co-Authored-By: Codex' <<< "$msg")" = 1 ]; then ok "a trailer already in msg.txt is not repeated"; else bad "a trailer already in msg.txt is not repeated" "$msg"; fi + +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run --add created.txt) && s=0 || s=$? +expect "codex run --add commits a new file named" 0 'committed' "$out" "$s" +if [ "$(git -C "$cxrepo" show --name-only --format= HEAD | sort | paste -sd' ')" = "created.txt file.txt" ]; then ok "--add puts the named file in the commit"; else bad "--add puts the named file in the commit"; fi + +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run --no-push) && s=0 || s=$? +expect "codex run --no-push commits and stops" 0 'committed' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 1 ] && [ ! -e "$STUB_GIT_ARGS" ] && [ ! -e "$STUB_GH_CALLS" ]; then ok "--no-push: a commit, no push, no pull request"; else bad "--no-push: a commit, no push, no pull request"; fi + +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run --no-pr) && s=0 || s=$? +expect "codex run --no-pr pushes and stops" 0 'pushed work' "$out" "$s" +if [ -e "$STUB_GIT_ARGS" ] && [ ! -e "$STUB_GH_CALLS" ]; then ok "--no-pr: a push, no pull request"; else bad "--no-pr: a push, no pull request"; fi + +cx_reset +out=$(STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "with no edits codex run makes no commit and exits 1" 1 'nothing to commit' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 0 ] && [ ! -e "$STUB_GIT_ARGS" ] && [ ! -e "$STUB_GH_CALLS" ]; then ok "no edits: no commit, no push, no pull request"; else bad "no edits: no commit, no push, no pull request"; fi + +cx_reset +out=$(STUB_CODEX_EDIT=1 cx_run) && s=0 || s=$? +expect "codex run refuses when codex wrote no msg.txt" 1 'wrote no msg.txt' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 0 ]; then ok "no msg.txt: no commit"; else bad "no msg.txt: no commit"; fi +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_NO_PR=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "codex run refuses when codex wrote no pr.md" 1 'wrote no pr.md' "$out" "$s" +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_EXIT=4 STUB_CODEX_MSG=$work/msg-plain.txt cx_run) && s=0 || s=$? +expect "codex run does not commit after a failed codex run" 4 'codex exit 4' "$out" "$s" +if [ "$(git -C "$cxrepo" rev-list --count "$base..HEAD")" = 0 ]; then ok "a failed codex run: no commit"; else bad "a failed codex run: no commit"; fi +cx_reset +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run --add msg.txt) && s=0 || s=$? +expect "codex run refuses --add msg.txt" 2 'read, never committed' "$out" "$s" +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt cx_run --add ../outside) && s=0 || s=$? +expect "codex run refuses --add outside the clone" 2 'inside the clone' "$out" "$s" +out=$(STUB_CODEX_EDIT=1 STUB_CODEX_MSG=$work/msg-plain.txt PATH=$work/gitstub:$work/ghbin:$PATH "$jpdev" codex run --brief "$work/brief.md" --model m --effort low --dir "$cxrepo" 2>&1) && s=0 || s=$? +expect "codex run needs --repo to open the pull request" 2 '--repo OWNER/NAME is needed' "$out" "$s" +if [ ! -e "$STUB_CODEX_ARGS" ]; then ok "a run refused for its flags did not start codex"; else bad "a run refused for its flags did not start codex"; fi +git -C "$cxrepo" switch -q main +out=$(STUB_CODEX_EDIT=1 cx_run) && s=0 || s=$? +expect "codex run refuses to commit on main" 2 'make a work branch first' "$out" "$s" +if [ ! -e "$STUB_CODEX_ARGS" ]; then ok "a run refused for the branch did not start codex"; else bad "a run refused for the branch did not start codex"; fi +git -C "$cxrepo" switch -q work +cx_reset +printf 'stale\n' > "$cxrepo/msg.txt" +out=$(STUB_CODEX_EDIT=1 cx_run --no-push) && s=0 || s=$? +expect "a msg.txt left by an earlier run is not used" 1 'wrote no msg.txt' "$out" "$s" + +# the brief template +out=$("$jpdev" codex brief-template 2>&1) && s=0 || s=$? +if [ "$out" = "$(cat "$kit/templates/codex-brief.md")" ]; then ok "brief-template prints templates/codex-brief.md"; else bad "brief-template prints templates/codex-brief.md"; fi +expect "the brief template tells codex to write msg.txt" 0 'msg\.txt' "$out" "$s" +expect "the brief template tells codex to write pr.md" 0 'pr\.md' "$out" "$s" +expect "the brief template tells codex not to commit" 0 'do not run .git commit.' "$out" "$s" +out=$("$jpdev" codex check-brief "$kit/templates/codex-brief.md" 2>&1) && s=0 || s=$? +expect "the brief template passes check-brief" 0 'no word of the filter list' "$out" "$s" +unset STUB_GIT_ARGS STUB_GH_CALLS STUB_GH_ARGS STUB_GH_BODY GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL +PATH=$oldpath +unset STUB_CODEX_ARGS STUB_CODEX_CWD STUB_CODEX_STDIN + # --- env printf 'JPDEV_NODE_RSS_MB=4321\nJPDEV_GO_TIMEOUT_S=99\n' > "$XDG_CONFIG_HOME/jpdev.env.tmp" mkdir -p "$XDG_CONFIG_HOME/jpdev"; mv "$XDG_CONFIG_HOME/jpdev.env.tmp" "$XDG_CONFIG_HOME/jpdev/env" @@ -326,6 +678,7 @@ out=$(JPDEV_GO_TIMEOUT_S=55 "$jpdev" env 2>&1) && s=0 || s=$? expect "env reports a config-file value as such" 0 'JPDEV_NODE_RSS_MB +4321 +config file' "$out" "$s" expect "env lets the environment win over the config file" 0 'JPDEV_GO_TIMEOUT_S +55 +environment' "$out" "$s" expect "env reports a default as such" 0 'JPDEV_PY_ULIMIT_KB +4000000 +default' "$out" "$s" +expect "env reports the codex sandbox default" 0 'JPDEV_CODEX_SANDBOX +workspace-write +default' "$out" "$s" rm -f "$XDG_CONFIG_HOME/jpdev/env" # --- setup, on a local origin