feat(cli): check skill downloads against the published tree hash

.agents/plans/skill-evals-and-packages.plan.md

@@ -7,16 +7,16 @@ todos:
     status: in_progress
   - id: p1-tree-hash-spec
     content: "Write docs/SKILL_TREE_HASH.md (the exact tree-hash algorithm, path rules, sort order) with shared test vectors. Done when web computeTreeHash and the new CLI implementation pass the same vectors."
-    status: in_progress
+    status: completed
   - id: p1-hash-sweep
     content: "Add a read-only script that recomputes tree_hash from served bytes for every public skill version and lists mismatches. Done when production shows zero mismatches, or each mismatch has a recorded fix."
     status: in_progress
   - id: p1-cli-verify
     content: "Make agentvouch skill install and update recompute the tree hash after download and stop on mismatch. Also reject trees where a path is both a file and a folder (at upload in normalizeSkillTreeFiles, and at install). Done when tampered-download tests fail the install, clean installs pass, and the CLI passes the shared tree-hash vectors."
-    status: pending
+    status: completed
   - id: p1-version-pinned-reads
     content: "Add a read-only, version-pinned archive download (same entitlement rules as today for paid skills). Done when a runner can fetch version N of a free skill and the bytes match the stored tree_hash."
-    status: pending
+    status: in_progress
   - id: p1-static-checks
     content: "Add deterministic static checks over the full tree (Agent Skills frontmatter rules, hidden Unicode, remote-install instructions, URL inventory, secret patterns), cached by (tree_hash, checks_version) and shown as a display-only signal. Done when each rule has tests and mirrored skills show results."
     status: pending
@@ -635,3 +635,11 @@ Each phase is one or more small PRs off current `main`. P3 and later need their
 - 2026-10-25: Owner answers. The owner runs the production sweep on their Mac. The owner reviews and merges every PR. No evaluation model spending yet (D13): P2 code is built and tested with mock runs only, so the "one real run" part of `p2-eval-runner` waits for a budget. Recommended defaults approved for D1-D11 and D14. D12 (auto-uninstall) stays off. Still open: Q2 (v1 default is AgentVouch-only results), Q3, Q4, Q8.
 - 2026-10-24: New finding (not yet seen in data): `ingestTarArchive` enforces the upload limits (200 files, 5 MB), but mirror sync stores trees up to 512 files and 8 MB. `/raw?path=` and `/zip` read stored trees through it, so they would fail for a mirrored tree above the upload limits; `/archive` would not. The sweep reports such versions as `files_unavailable` with the limit message. If the production sweep finds any, fix the reader to accept the limits the tree was stored with.
 - 2026-11-04: Refreshed PR #262 against current `main` for the owner's authorized PR review and merge sweep. Resolved the `AGENTS.md` conflict by preserving the short working instructions and adding one task-specific pointer to the tree-hash spec; package integrity details live in `docs/AGENT_REFERENCE.md`. The root is 1,135 words versus 5,295 in this PR's former manual (79% less). Audited all 35 numbered safeguards: 33 are verbatim in the companion and two only replace old section-number references. Both approval-boundary sections are preserved verbatim. Routine tasks do not require reading all reference material. Python passed all 10 vectors and 9 invalid paths; the three focused Vitest suites passed 91 tests. The production sweep remains unrun, and CLI download enforcement remains future work.
+- 2026-11-04 (`p1-cli-verify`, `p1-version-pinned-reads`, branch `feat/cli-tree-hash-verify`, on `main` after #262 was squash-merged as `a5a9798`): Phase 1, second PR.
+  - CLI: new `packages/agentvouch-cli/src/lib/integrity.ts`. `skill install` and `skills update` check every download against the published hashes before they write: a tree download against `tree_hash`, and a single `SKILL.md` against its manifest entry and the manifest against `tree_hash`. On a mismatch the CLI re-reads the skill once (a new version can be published during the download), then stops and writes nothing. The install metadata records `tree_hash_algorithm` and `tree_hash_verified_at`. The CLI passes the shared vectors, so `p1-tree-hash-spec` is done.
+  - Divergence: the CLI also stops before any download or payment when the skill has no published `tree_hash` (or, for one file, no manifest entry). Without this, a paid install could take the payment and then refuse to write.
+  - Server: upload rejects a file/folder path conflict with a 400, and `putSkillTree` rejects it too. Mirror sync skips such a skill with a `path conflict` detail, like a size-limit violation, so one bad upstream skill does not make the daily mirror cron return 500. The new route `/api/skills/{id}/versions/{version}/archive` serves one stored version with the same access rules as `/archive`.
+  - Divergence from the 2026-10-24 note: stored trees are now read with the mirror limits (512 files, 8 MB) without waiting for the sweep, because the fix is small and the CLI download check needs the same limits. The CLI reads downloads with the same limits.
+  - Found and fixed: `downloadRaw` decoded `/raw` with `response.text()`, which drops a leading byte order mark, so a written `SKILL.md` could differ from the published bytes. The CLI now hashes and writes the exact bytes.
+  - Merge gate: merge only after the production sweep (`p1-hash-sweep`) shows zero mismatches, or each mismatch has a recorded fix. `p1-version-pinned-reads` stays in progress until a live fetch of version N of a free skill matches its stored `tree_hash`.
+  - Verified: `format:check`, lint, web and CLI typecheck, web Vitest (166 files, 1,613 tests), CLI Vitest (18 files, 173 tests), and the Python vectors. `next build --webpack` compiled and ran TypeScript, then stopped at the `/sitemap.xml` prerender without `DATABASE_URL`. Not verified: the production sweep, and a live fetch from the new version route.

docs/SKILL_TREE_HASH.md

@@ -38,7 +38,7 @@ A stored path obeys all of these rules. Upload enforces them (`normalizeSkillPat
 
 AgentVouch stores and hashes path text exactly as it was received, as UTF-8. It applies no Unicode normalization. Two names that look the same can be different paths: `café.md` written with `é` (U+00E9, NFC) and `café.md` written with `e` plus a combining accent (U+0065 U+0301, NFD). They give different tree hashes. Publishers should use NFC names.
 
-**Client rule:** a client must also reject a tree in which one path is a folder of another path, for example `docs` and `docs/a.md`. Such a tree cannot be written to a file system. Upload does not enforce this yet (see [Known gaps](#known-gaps-2026-10-24)).
+**No file/folder conflicts:** a tree must not contain a path that is also a folder of another path, for example `docs` and `docs/a.md`. Such a tree cannot be written to a file system. Upload rejects it with a 400, and mirror sync skips the skill. A tree stored before these checks can still have a conflict; the sweep reports it as `path_conflict`. A client must reject such a tree.
 
 ## Algorithm
 
@@ -113,7 +113,14 @@ The manifest digests are the output of step 1, so the tree hash can be computed
    - Compute the digest of the downloaded bytes. Compare it with the manifest entry for that path.
    - Compute the tree hash from the manifest digests. Compare it with `tree_hash`.
    - Stop if either check fails.
-4. Record the checked tree hash with the install. The CLI writes it to `<path>.agentvouch.json`.
+4. Record the checked tree hash with the install. The CLI writes it to `<path>.agentvouch.json` with `tree_hash_algorithm` and `tree_hash_verified_at`.
+5. **An older version:** download `/api/skills/{id}/versions/{version}/archive`, and compare with the `tree_hash` of that entry in the `versions` list of `GET /api/skills/{id}`. The response also carries `X-AgentVouch-Tree-Hash` and `X-AgentVouch-Skill-Version` headers. Do not use those headers as the expected value, because they arrive with the bytes they describe.
+
+The `agentvouch` CLI runs steps 1 to 4 on every install and update of a skill that AgentVouch stores:
+
+- It stops before it downloads or pays if the skill has no published `tree_hash`, or (for one file) no manifest entry for that file.
+- If a check fails after the download, it re-reads the skill once, because a new version may have been published during the download. Then it stops without writing anything.
+- Chain-only skills (ids that start with `chain-`) are not stored by AgentVouch and have no tree hash, so the CLI cannot check them.
 
 Note: `/raw` serves `SKILL.md` from `skill_versions.content` for every storage type. The stored tree must therefore hold the same `SKILL.md` bytes. The sweep below checks this.
 
@@ -180,7 +187,7 @@ Finding codes:
 | `missing_manifest`            | The version has no stored manifest                                         |
 | `invalid_manifest`            | The manifest is not an array of valid entries                              |
 | `manifest_tree_hash_mismatch` | The manifest digests give a different tree hash                            |
-| `files_unavailable`           | The stored bytes could not be read: for example a missing Blob archive, or a tree above the upload limits (see Known gaps) |
+| `files_unavailable`           | The stored bytes could not be read (for example, a missing Blob archive)   |
 | `bytes_tree_hash_mismatch`    | The stored bytes give a different tree hash                                |
 | `file_digest_mismatch`        | A file's bytes do not match its manifest digest                            |
 | `file_size_mismatch`          | A file's length does not match its manifest size                           |
@@ -196,9 +203,13 @@ Finding codes:
 
 A change to any step is a new algorithm with a new name, for example `agentvouch-tree-v2`. Stored hashes keep the version that made them. Never change `agentvouch-tree-v1` in place.
 
-## Known gaps (2026-10-24)
+## Known gaps (updated 2026-11-04)
 
-- Upload does not reject a tree where one path is also a folder of another path. The sweep reports such trees as `path_conflict`. The fix is planned with the CLI download check (Phase 1, second PR).
-- The CLI records `tree_hash` but does not check downloads against it yet (Phase 1, second PR).
 - The `/zip` builder sorts entries with `localeCompare`, which depends on the server's ICU data. Zip bytes can differ between servers, so do not pin zip bytes. The tree hash does not depend on archive bytes.
-- The stored-tar reader (`ingestTarArchive`) enforces the upload limits (200 files, 5 MB), but mirror sync stores trees up to 512 files and 8 MB. `/raw?path=` and `/zip` use that reader, so they cannot serve a mirrored tree above the upload limits. The sweep reports such a version as `files_unavailable` with the limit message. `/archive` is not affected, because it serves the stored tar bytes directly.
+
+Fixed in Phase 1, second PR (2026-11-04):
+
+- Upload now rejects a tree where one path is also a folder of another path, and mirror sync skips such a skill.
+- The CLI now checks every download against the published hashes before it writes.
+- Stored trees are now read with the mirror limits (512 files, 8 MB) instead of the upload limits (200 files, 5 MB). Before this, `/raw?path=`, `/zip`, and CLI tree installs failed for a mirrored tree above the upload limits. The CLI also reads downloads with the mirror limits.
+- The CLI now hashes and writes the exact bytes of `/raw` responses. Before, it decoded them as text, which drops a leading byte order mark, so a written `SKILL.md` could differ from the published bytes.

packages/agentvouch-cli/README.md

@@ -103,6 +103,27 @@ agentvouch check "$SKILL_DIR" --fail-on avoid || {
 Text files only: the CLI sends each file as UTF-8, so the `tree_hash` of a
 skill that has binary assets can differ from the published hash.
 
+### Download checks
+
+`skill install` and `skills update` check each download before they write
+anything:
+
+- For a skill folder, the CLI recomputes the tree hash (`agentvouch-tree-v1`,
+  see [docs/SKILL_TREE_HASH.md](https://github.com/dirtybits/agentvouch/blob/main/docs/SKILL_TREE_HASH.md))
+  and compares it with the published `tree_hash`.
+- For a single `SKILL.md`, it compares the file's SHA-256 with the published
+  file list, and checks that list against `tree_hash`.
+- If the skill has no published tree hash, the CLI stops before it downloads or
+  pays.
+- If a check fails after the download, the CLI writes nothing and exits with an
+  error. For a paid skill your access is kept, so you can run the command again.
+- On success, `<path>.agentvouch.json` records `tree_hash`,
+  `tree_hash_algorithm`, and `tree_hash_verified_at`.
+
+Chain-only skills (ids that start with `chain-`) have no tree hash, so the CLI
+cannot check them. A matching hash proves that the bytes did not change. It
+does not prove that the skill is safe.
+
 ### Agents
 
 ```bash

packages/agentvouch-cli/src/cli.ts

@@ -422,6 +422,9 @@ export function buildProgram(): Command {
                 ...(typeof result.filesWritten === "number"
                   ? [`files_written: ${result.filesWritten}`]
                   : []),
+                ...(result.treeHash
+                  ? [`tree_hash_verified: ${result.treeHash}`]
+                  : []),
                 ...(result.dryRun ? ["dry_run: true"] : []),
               ]
             );

packages/agentvouch-cli/src/lib/archive.ts

@@ -7,10 +7,31 @@ const MAX_SKILL_CONTENT_BYTES = 256 * 1024;
 const MAX_SKILL_FILE_BYTES = 1024 * 1024;
 const MAX_SKILL_TREE_BYTES = 5 * 1024 * 1024;
 const MAX_SKILL_TREE_FILES = 200;
+// AgentVouch stores mirrored trees up to these limits
+// (web/lib/skillDraft.ts MIRROR_MAX_SKILL_TREE_*), so downloads may be this large.
+const MAX_DOWNLOADED_TREE_BYTES = 8 * 1024 * 1024;
+const MAX_DOWNLOADED_TREE_FILES = 512;
 const TAR_BLOCK_SIZE = 512;
 const SKIP_DIRECTORY_NAMES = new Set([".git", "node_modules"]);
 const SKIP_FILE_NAMES = new Set([".DS_Store"]);
 
+export interface TreeCaps {
+  maxFiles: number;
+  maxBytes: number;
+}
+
+/** Limits for trees the CLI publishes. */
+export const UPLOAD_TREE_CAPS: TreeCaps = {
+  maxFiles: MAX_SKILL_TREE_FILES,
+  maxBytes: MAX_SKILL_TREE_BYTES,
+};
+
+/** Limits for trees the CLI downloads. */
+export const DOWNLOAD_TREE_CAPS: TreeCaps = {
+  maxFiles: MAX_DOWNLOADED_TREE_FILES,
+  maxBytes: MAX_DOWNLOADED_TREE_BYTES,
+};
+
 export interface SkillTreeInputFile {
   path: string;
   content: Buffer;
@@ -42,14 +63,15 @@ function normalizeSkillPath(filePath: string): string {
 }
 
 function toNormalizedFiles(
-  files: SkillTreeInputFile[]
+  files: SkillTreeInputFile[],
+  caps: TreeCaps = UPLOAD_TREE_CAPS
 ): Array<SkillTreeInputFile & { bytes: Buffer }> {
   if (files.length === 0) {
     throw new CliError("Skill tree must include SKILL.md.");
   }
-  if (files.length > MAX_SKILL_TREE_FILES) {
+  if (files.length > caps.maxFiles) {
     throw new CliError(
-      `Skill tree has ${files.length} files, exceeds cap of ${MAX_SKILL_TREE_FILES}.`
+      `Skill tree has ${files.length} files, exceeds cap of ${caps.maxFiles}.`
     );
   }
 
@@ -71,9 +93,9 @@ function toNormalizedFiles(
       );
     }
     totalBytes += bytes.byteLength;
-    if (totalBytes > MAX_SKILL_TREE_BYTES) {
+    if (totalBytes > caps.maxBytes) {
       throw new CliError(
-        `Skill tree is ${totalBytes} bytes, exceeds cap of ${MAX_SKILL_TREE_BYTES} bytes.`
+        `Skill tree is ${totalBytes} bytes, exceeds cap of ${caps.maxBytes} bytes.`
       );
     }
 
@@ -196,7 +218,10 @@ function isZeroBlock(block: Buffer): boolean {
   return block.every((byte) => byte === 0);
 }
 
-export function ingestTarArchive(bytes: Buffer): SkillTreeInputFile[] {
+export function ingestTarArchive(
+  bytes: Buffer,
+  caps: TreeCaps = UPLOAD_TREE_CAPS
+): SkillTreeInputFile[] {
   const files: SkillTreeInputFile[] = [];
   let offset = 0;
   let totalBytes = 0;
@@ -216,14 +241,12 @@ export function ingestTarArchive(bytes: Buffer): SkillTreeInputFile[] {
     const filePath = normalizeSkillPath(prefix ? `${prefix}/${name}` : name);
     const size = readOctal(header, 124, 12);
     totalBytes += size;
-    if (files.length + 1 > MAX_SKILL_TREE_FILES) {
-      throw new CliError(
-        `Skill tree exceeds cap of ${MAX_SKILL_TREE_FILES} files.`
-      );
+    if (files.length + 1 > caps.maxFiles) {
+      throw new CliError(`Skill tree exceeds cap of ${caps.maxFiles} files.`);
     }
-    if (totalBytes > MAX_SKILL_TREE_BYTES) {
+    if (totalBytes > caps.maxBytes) {
       throw new CliError(
-        `Skill tree is ${totalBytes} bytes, exceeds cap of ${MAX_SKILL_TREE_BYTES} bytes.`
+        `Skill tree is ${totalBytes} bytes, exceeds cap of ${caps.maxBytes} bytes.`
       );
     }
     if (offset + size > bytes.byteLength) {
@@ -238,7 +261,7 @@ export function ingestTarArchive(bytes: Buffer): SkillTreeInputFile[] {
     offset += (TAR_BLOCK_SIZE - (size % TAR_BLOCK_SIZE)) % TAR_BLOCK_SIZE;
   }
 
-  return toNormalizedFiles(files);
+  return toNormalizedFiles(files, caps);
 }
 
 /**
@@ -290,12 +313,11 @@ export async function prepareSkillUploadFromPath(
   };
 }
 
-export async function writeTarArchiveToDirectory(
+/** Write files that were already parsed (and checked) to a folder. */
+export async function writeSkillFilesToDirectory(
   outputDir: string,
-  archive: Buffer
+  files: SkillTreeInputFile[]
 ): Promise<number> {
-  const files = ingestTarArchive(archive);
-
   for (const file of files) {
     const targetPath = path.join(outputDir, ...file.path.split("/"));
     await mkdir(path.dirname(targetPath), { recursive: true });
@@ -305,6 +327,16 @@ export async function writeTarArchiveToDirectory(
   return files.length;
 }
 
+export async function writeTarArchiveToDirectory(
+  outputDir: string,
+  archive: Buffer
+): Promise<number> {
+  return writeSkillFilesToDirectory(
+    outputDir,
+    ingestTarArchive(archive, DOWNLOAD_TREE_CAPS)
+  );
+}
+
 export function isDirectoryLikeOutput(outputPath: string): boolean {
   return outputPath.endsWith(path.sep) || path.extname(outputPath) === "";
 }

packages/agentvouch-cli/src/lib/fs.ts

@@ -34,3 +34,12 @@ export async function writeUtf8File(
   await mkdir(path.dirname(filePath), { recursive: true });
   await writeFile(filePath, contents, "utf8");
 }
+
+/** Write bytes exactly as given (no text decoding or encoding). */
+export async function writeBytesFile(
+  filePath: string,
+  contents: Uint8Array
+): Promise<void> {
+  await mkdir(path.dirname(filePath), { recursive: true });
+  await writeFile(filePath, contents);
+}

packages/agentvouch-cli/src/lib/http.ts

@@ -193,6 +193,8 @@ export interface DownloadResponse {
   ok: boolean;
   status: number;
   content?: string;
+  /** Exact response bytes for /raw. Hash these, not the decoded text. */
+  bytes?: Buffer;
   archive?: Buffer;
   error?: string;
   requirement?: PaymentRequirement;
@@ -511,10 +513,14 @@ export class AgentVouchApiClient {
       : undefined;
 
     if (response.ok) {
+      // Read bytes, not response.text(): text() drops a leading byte order
+      // mark, which would change the file hash.
+      const bytes = Buffer.from(await response.arrayBuffer());
       return {
         ok: true,
         status: response.status,
-        content: await response.text(),
+        content: bytes.toString("utf8"),
+        bytes,
         paymentResponse,
       };
     }