Skip to content

Fix doc drift in the trust model and restated conventions - #82

Merged
dmccoystephenson merged 2 commits into
mainfrom
fix/doc-accuracy-sweep-trust-model-and-conventions
Jul 29, 2026
Merged

Fix doc drift in the trust model and restated conventions#82
dmccoystephenson merged 2 commits into
mainfrom
fix/doc-accuracy-sweep-trust-model-and-conventions

Conversation

@dmccoystephenson

@dmccoystephenson dmccoystephenson commented Jul 29, 2026

Copy link
Copy Markdown
Member

Summary

A Stage A documentation-accuracy sweep — a full pass over every file in CLAUDE.md's "Documentation sources
of truth" table, verifying each claim against create-dev-loop.md and scripts/check_docs.py rather than
against memory. Three files had drifted; the fixes are doc-only.

SECURITY.md — the trust model understated what the target repo controls. CLAUDE.md requires this file
to match "what Step 2 of create-dev-loop.md actually reads from the target repo." It didn't:

  • Its list of reads omitted build files, linter/formatter configs, the PR template, documentation sources,
    and commit history — Step 2 reads all of them.
  • More importantly, it framed the consequence as cosmetic ("its stated identity, its reviewer, its branch
    conventions"). The sharper consequence was missing entirely: Step 4 derives COMPILE_CMD, TEST_CMD,
    LINT_CMD, and EXTERNAL_SIGNAL_CMD from the target repo's build files and CI workflows, and the
    generated skill executes them verbatim. DO_NOT_AUTO_MERGE — which the target repo also supplies —
    decides what that skill will refuse to merge without a human.
  • Its list of what the skill later does ("create branches, open PRs, and push commits") omitted running those
    derived commands and merging PRs (Phase 8 gh pr merge --squash).

For a document whose whole job is telling readers why to only run this against repos they trust, "the repo
you point this at chooses the shell commands the agent will run and which paths it may auto-merge" is the
load-bearing sentence, and it wasn't there.

CONTRIBUTING.md — restated conventions were incomplete. CLAUDE.md requires this file's restated
conventions to match it. Three were absent: repo-specific findings belong only in a generated skill and must
never be back-ported; Steps 3/5/6 are a load-bearing downstream interface whose changes need flagging in the
PR description; and the squash-merge/delete-branch and co-author-trailer conventions. The Steps 3/5/6 rule is
referenced through CLAUDE.md's own section rather than by adding a second hyperlink to the gardener repo —
see #79 for why.

RESEARCH.md — "Last reviewed" bumped to 2026-07-29 to reflect this sweep. All eight findings, their
citations, confidence levels, and Implementations entries were re-read; no inaccuracy found.

Filed rather than fixed here

Three findings from the sweep are code, not doc drift, so per the Stage A rule ("if the code is what's
wrong, file an issue and leave it for an implementation cycle") they are issues:

No open issues existed at triage, so nothing was deferred.

Research grounding

No finding applies. This is a documentation-accuracy correction; it touches no template placeholder, no Step
logic, and no phase definition. RESEARCH.md's only change is the review date — no finding, citation,
confidence level, or Implementations entry is added, removed, or reinterpreted.

Doc sync check

  • README.md's "What it does" Step list still matches create-dev-loop.md's Steps 1:1 — unchanged, and verified by scripts/check_docs.py
  • Every {{placeholder}} added or changed has a corresponding Step 4 substitution-table row — none added or changed
  • RESEARCH.md updated — review date only, per above

Test plan

  • python3 scripts/check_docs.py passes locally (placeholder/table coverage, README ↔ Steps 1:1, relative links). CI runs it on this PR.
  • Every new SECURITY.md claim traced to source: the read list against Step 2's file enumeration; COMPILE_CMD/TEST_CMD/LINT_CMD/EXTERNAL_SIGNAL_CMD/DO_NOT_AUTO_MERGE/SELF_REVIEW_RUBRIC against their Step 4 rows; "Phase 3 build-verification step" and "Phase 4 external-signal anchor" against where those placeholders actually appear in the embedded template; "merge PRs" against Phase 8's gh pr merge --squash.
  • Every new CONTRIBUTING.md line traced to the CLAUDE.md section it restates.
  • New relative link [CLAUDE.md](CLAUDE.md) resolves — covered by check_local_links.

Behavioral anchor: UNVERIFIED-not-applicable. create-dev-loop.md is untouched, as are the Step 4
substitution table and all Step 1/5–7 logic, so there is no generation behavior for a /create-dev-loop run
against a live repo to verify. Per the dev-loop's Phase 4 scope gate, that is the documented condition for
recording the anchor as not-applicable rather than required.


This PR description was drafted during a Gardener session (Stephenson-Software/gardener).

dmccoystephenson and others added 2 commits July 29, 2026 01:51
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric — scored adversarially against the diff and command output, not judgment.

External anchor. doc-consistency pass (7s) on head 53adf93 — run 30415288400. Behavioral anchor recorded UNVERIFIED-not-applicable: create-dev-loop.md, the Step 4 substitution table, and Step 1/5–7 logic are all untouched, which is the documented condition for not-applicable rather than required.

  • Scope: PASSgit diff --stat origin/main is CONTRIBUTING.md | +14/-1, RESEARCH.md | +1/-1, SECURITY.md | +32/-7. All three are rows in CLAUDE.md's documentation sources-of-truth table, which is exactly the sweep's scope. No formatting churn, no renames, no create-dev-loop.md edit smuggled into a docs cycle.
  • Tests-new: N/A — no public method, function, or script added. This repo has no automated test suite beyond scripts/check_docs.py, which is unmodified.
  • Tests-fix: PASS (empirically, not reasoned) — the drift is directly observable in the pre-diff text rather than inferred. SECURITY.md at c1ac200 enumerated the target-repo reads as "CLAUDE.md, CONTRIBUTING.md, README.md, CI configs, CODEOWNERS, and recent PR descriptions"; Step 2 of create-dev-loop.md additionally reads seven build-file types, linter and formatter configs, the PR template, documentation sources, and git log. The omission is checkable by reading the two lists side by side, and is gone after the change. Same for the CONTRIBUTING.md items: each added line maps to a named CLAUDE.md section that had no counterpart there.
  • Sibling structure: PASS — no new files created.
  • Sibling renames: PASS — no identifier renamed. The CONTRIBUTING.md additions are new bullets in an existing list, matching the surrounding bullet form.
  • Docs: PASS — checked the reverse direction for drift introduced by this PR. README.md's security pointer ("/create-dev-loop reads and acts on content from whatever repo you run it in") still matches the rewritten trust model; README.md's "What it does" Step list is untouched and still 1:1; RESEARCH.md's findings are unchanged apart from the review date.
  • Issue resolution: PASS — no Closes #N is claimed. README and CLAUDE.md link gardener as an open-source companion, but the repo is private #79/check_docs.py reads the Step 4 substitution table through a fixed 8000-byte window with only 231 bytes of headroom #80/CLAUDE.md's documentation sources-of-truth table omits .github/ISSUE_TEMPLATE/*.md #81 are deliberately left open; all three are code changes, and the Stage A rule is to file rather than fix them under a docs cycle.
  • Placeholder table completeness: PASS — verified by running the Phase 3 grep, not from memory: every {{TOKEN}} between the Step 3 fences has a Step 4 row (USED not DECLARED: []). No placeholder added or changed by this PR regardless.
  • Fence escaping intact: PASScreate-dev-loop.md is not in the diff; a scan of the embedded template body found zero unescaped fences.
  • README/Step 1:1: PASS — no Step added, removed, or renumbered; check_docs.py's check_readme_steps_sync confirms.
  • RESEARCH.md citation: N/A, stated explicitly — this PR does not change the template, so no finding applies. Called out in the PR body rather than left silent, per CLAUDE.md.

One item I fixed mid-review rather than passing. The first push described DO_NOT_AUTO_MERGE as deciding "what the generated skill refuses to merge without a human." That overstates target-repo control: Phase 8 enforces a universal baseline (.github/workflows/*, security/, >50-line deletions) that a crafted repo cannot shrink — it can only widen the list. Overstating attacker control in a trust model is still an inaccuracy, so 53adf93 corrected it to say so explicitly.

Two observations outside the diff hunks, folded in here rather than posted inline:

  • SECURITY.md:new — the executes-verbatim bullet names "its Phase 3 build-verification step and its Phase 4 external-signal anchor". The generated template also re-runs the same command via REVALIDATE_INSTRUCTION in Phase 6 and via TEST_CMD in the Phase 8 rebase edge case. The enumeration understates breadth rather than misstating it, so I left it — naming every site would date the sentence against phase-numbering changes.
  • scripts/check_docs.py:104check_local_links skips anything starting with http:///https://, which is why CI could not catch README and CLAUDE.md link gardener as an open-source companion, but the repo is private #79. Not in scope here; noted on that issue.

@dmccoystephenson
dmccoystephenson merged commit 8ba104d into main Jul 29, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the fix/doc-accuracy-sweep-trust-model-and-conventions branch July 29, 2026 01:53
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
* Fix doc drift in the trust model and restated conventions

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Clarify that DO_NOT_AUTO_MERGE can only widen the merge gate

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant