diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a40ca332..703d09ee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -70,4 +70,11 @@ jobs: - name: Test env: SYNCWHEEL_SOURCE_ROOT: ${{ github.workspace }}/syncwheel - run: pnpm test + run: node scripts/test-command-matrix.mjs "${{ runner.temp }}/test-command-matrix.json" + + - name: Upload command test matrix + if: always() + uses: actions/upload-artifact@v4 + with: + name: test-command-matrix + path: ${{ runner.temp }}/test-command-matrix.json diff --git a/docs/read-effects.md b/docs/read-effects.md new file mode 100644 index 00000000..81c954e7 --- /dev/null +++ b/docs/read-effects.md @@ -0,0 +1,27 @@ +# Status observation and explicit refresh + +`status` now observes local state and existing caches by default, regardless of cache TTL. `status --refresh` explicitly permits cache population, source fetches, snapshot maintenance, version discovery and composite member refresh. `--offline` wins over `--refresh`. Missing or stale evidence remains visible in the status protocol instead of causing an implicit refresh. Useful cached Git snapshots and local sources still support pending-work inspection; an absent cached snapshot returns a named refresh requirement. + +## Consumer behavior change and cache refresh responsibility + +Previously, a status read could refresh expired member and package-version caches automatically. It no longer does. The default `fleet-diagnostics` wrapper forwards an observation to `status`; after the member or version TTL expires it therefore shows stale evidence (`STALE`, or a degraded status derived from stale package metadata) until an explicit refresh succeeds. Repeated diagnostics reads do not make those caches fresh. Missing caches stay missing and are reported as unavailable. + +The operator or caller requesting fresh evidence now owns the refresh: run `agentwheel status` with the **same workspace/fleet and profile selectors** plus `--refresh`, or explicitly opt into the diagnostics caller's existing `--refresh` option. The refresh path updates the member-report and version-index caches; subsequent default reads consume those caches without extending their freshness. No background refresher, timer, or new automated refresh owner is added here. Any change to fleet-control consumers or their refresh schedule is a separate maintainer decision and follow-up; this PR does not change fleet-control. + +For example, `agentwheel status --fleet example-fleet --profile example-profile --refresh --json` refreshes that selected scope. `--offline` must be omitted when refreshing. Existing mutating commands retain their own cache behavior; this change concerns the read route. + +## Effect policy + +The AsyncLocalStorage scope in `src/effects/read-policy.ts` centralizes this distinction across graph sources, registry/version indexes, member reports, and cache writers. Observation never creates cache leases or maintenance locks and does not prune snapshots. Incomplete checkout directories are preserved even during refresh: repair is a separate explicit operation. The error identifies the exact checkout directory to move into a new backup location outside its cache directory. Confirm the original checkout path is absent, then rerun the same scoped command with `--refresh` to clone it again. Do not overwrite existing backups. No repair command or automatic move/removal is provided. Git status disables optional index locks. The optional update notifier does not fetch or persist on status reads. + +Observation supports built-in local targets and cached composite reports. Unqualified programmatic adapters, source drivers other than local/Git, and direct remote transports refuse before execution; explicit refresh retains their existing admission and execution paths. This deliberately avoids promising zero effects for arbitrary executable code. Other CLI read/preview verbs have not yet been qualified by this batch. + +Diagnostics consumers of the versioned `status --json` protocol inherit this policy. Tests exercise a synthetic child-process consumer, not a private fleet configuration. No fleet diagnostics source is changed by this PR. + +`scripts/agentwheel-local.mjs` is the generic source-owned local launcher. It requires a prebuilt, current `dist/index.js`, or refuses with an explicit build instruction. It never runs a package manager, acquires a build lock, or builds implicitly, for any verb. `AGENTWHEEL_REPO_DIR` selects a checkout; the default is the repository containing the launcher. This PR does not replace any installed wrapper or retarget a runtime launcher. + +## Evidence and limits + +The B0a effects helper is reused from PR #201; this PR is based on that branch. Before the product changes, synthetic status fixtures reproduced clone/fetch/tag queries, persistent cache leases and version-index writes, global pruning when a newer source commit creates a snapshot, and removal of a sentinel from an incomplete checkout. A separately isolated legacy-launcher probe reproduced one implicit build on a status invocation with a dummy builder. That launcher was unversioned: the replacement above establishes a source-owned implementation without copying host-specific defaults into public source. + +The fixture snapshots synthetic HOME, workspace, cache, Git metadata and temp before/after each invocation. A child preload records process calls and rejects Node network APIs; only its exact evidence file is excluded from tree comparison. Tests cover cold/warm/offline/refused reads, a diagnostics protocol consumer, wrapper admission, and explicitly bounded refresh writes. External Git network access is checked through command arguments; this is an application-boundary trace, not a kernel syscall sandbox. Concurrent refresh/prune versus a lease-free observer remains a separate cache-lifecycle qualification concern: observation may report missing evidence but cannot repair or delete it. diff --git a/docs/testing-invariants.md b/docs/testing-invariants.md new file mode 100644 index 00000000..b89031b3 --- /dev/null +++ b/docs/testing-invariants.md @@ -0,0 +1,22 @@ +# Invariant baseline and command attribution + +Run `node scripts/test-command-matrix.mjs [output.json]` from the repository root after installing dependencies. Without an output argument, the report is saved under the operating system temporary directory. Parser scratch files also live there. The script runs the entire Vitest suite with the default reporter streamed to stdout/stderr; failures remain in the output and yield a nonzero exit. It loads the actual Commander tree with only its executable entrypoint removed, including root, groups and hidden commands. No command handler runs during inventory. + +`scripts/test-command-map.json` currently links only three commands and is deliberately conservative: every link names exactly one runner-registered file/full-name pair and documents why that test covers the command. Imports and shared files do not imply command coverage. Add audited links as coverage is established; an unattributed test is not evidence of absent behavior. Unknown commands, missing or ambiguous cases, duplicate IDs and registration/result mismatches refuse generation. Parameterized cases remain individual runner IDs. Per-command counts overlap; only the union of IDs is a suite total. IDs are Vitest IDs, stable for the same runner/project/module/test order. The separate JSON reporter reconciles the overall count. + +The report records input-byte and parser hashes, runner and Node versions. It excludes timing, absolute paths and stack traces. `unattributedCommands` lists commands with no declared test links; this is not a claim that their behavior is untested. Unattributed tests are explicitly listed to reconcile the full runner inventory. `it.fails` outcomes count as runner passes and carry `expectedFailure: true`; this is not proof that the invariant holds. Skipped and pending outcomes are separate from executed cases. Generated reports are observations of one environment and are not committed as a portable status baseline. + +## Effects harness + +`test/helpers/effects.ts` records file bytes in base64, permission bits, directories and symlink targets without following links. Explicit manifest paths record complete parsed claims as well as original bytes. With `git: true`, the snapshot also records the staged index (semantic entries and raw bytes), refs and symbolic HEAD; the filesystem tree includes raw Git metadata. Locks under the fixture root are included. Filesystem atime, mtime, ctime and birthtime are not recorded; all listed content and permission fields are compared exactly. There are no default path exclusions. A caller may exclude only an exact file or symlink path with a nonempty reason; directories are always traversed and manifest claims cannot be excluded. Compare snapshots with `diffEffects`; unknown changed paths remain visible. This is an isolated filesystem observer, not a syscall/network/process tracer. + +## Invariant cases + +The invariant tests use synthetic fixtures and no live targets. I2 now uses ordinary regression tests; I5 remains an expected failure. Each expected-failure test has an adjacent ordinary test that checks the exact current defect, so a broken setup cannot masquerade as an expected failure. A product fix must remove that marker in the same change: an unexpected pass fails CI. Do not replace the marker with skip or todo. + +| Invariant | Case | Observed defect | +| --- | --- | --- | +| I2 | `read_missing_git_preserves_cache` | Fixed: an incomplete cache checkout is preserved and refuses implicit repair. | +| I5 | `stale_merge_metadata_rejected_in_all_paths` | Preview accepts unchanged source with stale contribution metadata while apply rejects the contribution. | + +An unreachable source during pin preflight produces a warning by design, not a defect: it does not prove a new artifact collision. The existing pin-preflight tests retain that intended behavior. diff --git a/scripts/agentwheel-local.mjs b/scripts/agentwheel-local.mjs new file mode 100644 index 00000000..be416ef3 --- /dev/null +++ b/scripts/agentwheel-local.mjs @@ -0,0 +1,23 @@ +#!/usr/bin/env node +import { readdirSync, statSync } from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { spawnSync } from "node:child_process"; + +const root = resolve(process.env.AGENTWHEEL_REPO_DIR ?? join(dirname(fileURLToPath(import.meta.url)), "..")); +const entry = join(root,"dist/index.js"); +function newest(path) { + let stat; + try { stat = statSync(path); } catch { return 0; } + return stat.isDirectory() ? Math.max(stat.mtimeMs,...readdirSync(path).map(name => newest(join(path,name)))) : stat.mtimeMs; +} +let built = 0; +try { built = statSync(entry).mtimeMs; } catch { /* Missing builds are refused below. */ } +if (!built || ["src","package.json","pnpm-lock.yaml","tsconfig.json","tsup.config.ts"].some(path => newest(join(root,path)) > built)) { + console.error("agentwheel: built CLI is missing or stale. Build explicitly with pnpm build in the selected checkout; this wrapper never builds implicitly."); + process.exitCode = 127; +} else { + const result = spawnSync(process.execPath,[entry,...process.argv.slice(2)],{ stdio: "inherit" }); + if (result.error) console.error(result.error.message); + process.exitCode = result.status ?? 1; +} diff --git a/scripts/test-command-map.json b/scripts/test-command-map.json new file mode 100644 index 00000000..8e093074 --- /dev/null +++ b/scripts/test-command-map.json @@ -0,0 +1,23 @@ +{ + "schemaVersion": 1, + "links": [ + { + "file": "test/cli-verb-redesign.test.ts", + "name": "CLI verb redesign > previews and applies Git cache pruning", + "commands": ["cache prune"], + "evidence": "Runs cache prune in preview and apply modes and checks retained cache entries." + }, + { + "file": "test/cli-verb-redesign.test.ts", + "name": "CLI verb redesign > drafts a registry publish submission from a GitHub URL without writing workspace state", + "commands": ["registry publish"], + "evidence": "Invokes registry publish and checks its draft and workspace effects." + }, + { + "file": "test/cli-verb-redesign.test.ts", + "name": "CLI verb redesign > doctor suggests the Copilot user companion skill install without writing files", + "commands": ["doctor"], + "evidence": "Invokes doctor and checks the suggested install and unchanged tree." + } + ] +} diff --git a/scripts/test-command-matrix.mjs b/scripts/test-command-matrix.mjs new file mode 100644 index 00000000..16f21e11 --- /dev/null +++ b/scripts/test-command-matrix.mjs @@ -0,0 +1,137 @@ +import { createHash } from "node:crypto"; +import { readFile, writeFile, mkdtemp, rm } from "node:fs/promises"; +import { join, relative, resolve } from "node:path"; +import { tmpdir } from "node:os"; +import { pathToFileURL } from "node:url"; +import { spawnSync, execFileSync } from "node:child_process"; +import { createRequire } from "node:module"; + + +const root = process.cwd(); +const require = createRequire(import.meta.url); +// Use the bundler shipped with the pinned, direct tsup development dependency. +const { build } = createRequire(require.resolve("tsup/package.json"))("esbuild"); +const compare = (a, b) => a < b ? -1 : a > b ? 1 : 0; +const hash = bytes => createHash("sha256").update(bytes).digest("hex"); +const read = path => readFile(join(root, path), "utf8"); + +export function counts(tests) { + const pass = tests.filter(t => t.status === "passed").length; + const fail = tests.filter(t => t.status === "failed").length; + return { registered: tests.length, executed: pass + fail, pass, fail, + skipped: tests.filter(t => t.status === "skipped").length, + pending: tests.filter(t => t.status === "pending").length }; +} + +export function reconcile(commands, registered, results, links) { + const byId = new Map(registered.map(t => [t.id, t])); + const resultIds = new Set(results.map(t => t.id)); + if (byId.size !== registered.length || resultIds.size !== results.length + || byId.size !== resultIds.size || [...byId.keys()].some(id => !resultIds.has(id))) { + throw new Error("Registered and executed runner IDs do not reconcile"); + } + const attributed = new Map(commands.map(command => [command, new Set()])); + const assigned = new Set(); + const selectors = new Set(); + for (const link of links) { + const key = `${link.file}::${link.name}`; + if (selectors.has(key) || !link.evidence?.trim() || !Array.isArray(link.commands) || !link.commands.length) throw new Error(`Invalid mapping: ${key}`); + selectors.add(key); + const matches = registered.filter(t => t.file === link.file && t.name === link.name); + if (matches.length !== 1) throw new Error(`Mapping must match exactly one registered test: ${key} (${matches.length})`); + for (const command of link.commands) { + if (!attributed.has(command)) throw new Error(`Unknown parser command: ${command}`); + attributed.get(command).add(matches[0].id); + assigned.add(matches[0].id); + } + } + const rows = commands.map(command => { + const tests = results.filter(t => attributed.get(command).has(t.id)).sort((a,b) => compare(a.id,b.id)); + return { command, ...counts(tests), tests }; + }); + return { rows, uniqueAttributed: assigned.size, + unattributed: results.filter(t => !assigned.has(t.id)).sort((a,b) => compare(a.id,b.id)) }; +} + +export async function parserCommands(scratch) { + const source = await read("src/cli/index.ts"); + const marker = "try {\n await main();"; + if (source.split(marker).length !== 2) throw new Error("CLI entry boundary changed; inspect inventory loader"); + // Load the actual Commander tree, replacing only the executable entrypoint. + const code = `${source.slice(0, source.indexOf(marker))}\nexport { program };\n`; + const output = join(scratch, "parser.mjs"); + await build({ stdin: { contents: code, resolveDir: join(root,"src/cli"), sourcefile: "index.ts", loader: "ts" }, + bundle: true, plugins: [{ name: "resolve-temporary-parser-imports", setup(context) { + context.onResolve({ filter: /^[^./]/ }, args => ({ + path: args.path.startsWith("node:") ? args.path : import.meta.resolve(args.path), external: true, + })); + } }], platform: "node", format: "esm", outfile: output, write: true }); + const { program } = await import(pathToFileURL(output)); + const commands = [program.name()]; + function visit(parent, prefix) { + for (const command of parent.commands) { + const path = [...prefix, command.name()]; + commands.push(path.join(" ")); + visit(command, path); + } + } + visit(program, []); + if (new Set(commands).size !== commands.length) throw new Error("Duplicate parser command path"); + return commands.sort(compare); +} + +async function sourceReceipt() { + // Hash exact source/test/script/config bytes; no absolute paths, timestamps or self-referential output. + const tracked = execFileSync("git", ["ls-files", "-z"], { cwd: root, encoding: "utf8" }).split("\0").filter(Boolean); + const paths = tracked.filter(path => /^(src|test|scripts)\//.test(path) + || ["package.json", "pnpm-lock.yaml", "vitest.config.ts", "tsconfig.json"].includes(path)); + const files = await Promise.all(paths.sort(compare).map(async path => [path, hash(await readFile(join(root,path)))])); + return { inputDigest: hash(JSON.stringify(files.sort((a,b) => compare(a[0],b[0])))), + parserSha256: hash(await read("src/cli/index.ts")), + vitest: require("vitest/package.json").version, + node: process.versions.node }; +} + +export function runTests(report, json) { + return spawnSync("pnpm", ["test", "--maxWorkers=2", "--reporter=default", "--reporter=json", + "--reporter=./scripts/test-command-reporter.mjs", `--outputFile=${json}`], { + cwd: root, env: { ...process.env, COMMAND_MATRIX_REPORT: report }, stdio: "inherit", + }); +} + +export async function matrixDestination(path) { + return path ? resolve(path) : join(await mkdtemp(join(tmpdir(), "command-matrix-output-")), "matrix.json"); +} + +async function main() { + const scratch = await mkdtemp(join(tmpdir(), "command-matrix-parser-")); + const reportRoot = await mkdtemp(join(tmpdir(), "command-matrix-")); + try { + const commands = await parserCommands(scratch); + const report = join(reportRoot,"runner.json"); + const json = join(reportRoot,"vitest.json"); + const runner = runTests(report, json); + if (runner.error) throw runner.error; + const observed = JSON.parse(await readFile(report,"utf8")); + const official = JSON.parse(await readFile(json,"utf8")); + const officialTests = official.testResults.flatMap(file => file.assertionResults); + if (officialTests.length !== observed.tests.length) throw new Error("JSON reporter test count mismatch"); + const mapping = JSON.parse(await read("scripts/test-command-map.json")); + if (mapping.schemaVersion !== 1) throw new Error("Unsupported mapping schema"); + const matrix = reconcile(commands, observed.registered, observed.tests, mapping.links); + const output = { schemaVersion: 1, receipt: await sourceReceipt(), + commandCount: commands.length, unattributedCommands: matrix.rows.filter(row => !row.registered).map(row => row.command), + totals: counts(observed.tests), uniqueAttributed: matrix.uniqueAttributed, + runnerExitCode: runner.status, moduleErrors: observed.moduleErrors, unhandledErrors: observed.unhandledErrors, + commands: matrix.rows, unattributed: matrix.unattributed }; + const destination = await matrixDestination(process.argv[2]); + await writeFile(destination, `${JSON.stringify(output,null,2)}\n`); + console.log(JSON.stringify({ commandCount: output.commandCount, unattributedCommands: output.unattributedCommands.length, + ...output.totals, uniqueAttributed: output.uniqueAttributed, output: relative(root,destination) })); + if (runner.status !== 0 || observed.moduleErrors.length || observed.unhandledErrors) process.exitCode = 1; + } finally { + await rm(scratch, { recursive: true, force: true }); + await rm(reportRoot, { recursive: true, force: true }); + } +} +if (process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) await main(); diff --git a/scripts/test-command-reporter.mjs b/scripts/test-command-reporter.mjs new file mode 100644 index 00000000..632e4ffd --- /dev/null +++ b/scripts/test-command-reporter.mjs @@ -0,0 +1,28 @@ +import { writeFileSync } from "node:fs"; + +// Uses runner-owned IDs, including parameter expansion, rather than source-text guesses. +export default class CommandMatrixReporter { + registered = new Map(); + onTestModuleCollected(module) { + for (const test of module.children.allTests()) { + if (this.registered.has(test.id)) throw new Error(`Duplicate runner test ID: ${test.id}`); + this.registered.set(test.id, { + id: test.id, file: module.relativeModuleId, name: test.fullName, + expectedFailure: test.options.fails === true, + }); + } + } + onTestRunEnd(modules, errors, reason) { + const tests = []; + for (const module of modules) for (const test of module.children.allTests()) { + const registered = this.registered.get(test.id); + if (!registered) throw new Error(`Unregistered runner test: ${test.id}`); + tests.push({ ...registered, status: test.result().state }); + } + writeFileSync(process.env.COMMAND_MATRIX_REPORT, JSON.stringify({ + registered: [...this.registered.values()], tests, reason, + moduleErrors: modules.filter(module => !module.ok()).map(module => module.relativeModuleId), + unhandledErrors: errors.length, + })); + } +} diff --git a/src/adapters/resolve.ts b/src/adapters/resolve.ts index 8ea3a963..b696dc9e 100644 --- a/src/adapters/resolve.ts +++ b/src/adapters/resolve.ts @@ -1,4 +1,5 @@ import { resolve } from "node:path"; +import { isObservation } from "../effects/read-policy.js"; import type { AdapterConfig } from "../model/adapter.js"; import { loadAdapterConfig } from "../model/adapter.js"; import { getAdapter } from "./index.js"; @@ -15,6 +16,7 @@ export interface ResolveAdapterOptions { export async function resolveAdapter(options: ResolveAdapterOptions): Promise { if (options.adapterModule) { + if (isObservation()) throw new Error("Observation refuses unqualified adapter code; use --refresh explicitly."); const modulePath = resolve(options.baseDir ?? process.cwd(), options.adapterModule); const adapter = await loadProgrammaticAdapter(modulePath, { allowCode: options.allowAdapterCode === true }); options.warn?.(`WARNING: loaded local adapter code ${adapter.programmatic?.modulePath} (${adapter.programmatic?.hash})`); diff --git a/src/cli/index.ts b/src/cli/index.ts index 7acac5bf..8d597f02 100644 --- a/src/cli/index.ts +++ b/src/cli/index.ts @@ -5,6 +5,7 @@ import { homedir, tmpdir } from "node:os"; import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; import { fileURLToPath } from "node:url"; import { Command } from "commander"; +import { withReadEffects } from "../effects/read-policy.js"; import { resolveAdapter } from "../adapters/resolve.js"; import { installRootForAdapterInstallationType, @@ -1677,7 +1678,7 @@ program .option("--offline", "use cached version and member status even when stale", false) .option("--refresh", "refresh package and member status regardless of TTL", false) .option("--recover-legacy-state", "inspect stable state while preserving invalid historical state", false) - .action(async (options) => { + .action(async (options) => withReadEffects(options, async () => { const normalizedOptions = normalizeRuntimeScopeOptions(options); const composite = await resolveSelectedCompositeProfile(normalizedOptions); if (composite) { @@ -1699,7 +1700,7 @@ program for (const target of targets) { await printStatus(target, normalizedOptions); } - }); + })); const journalCommand = program .command("journal") diff --git a/src/cli/update-check.ts b/src/cli/update-check.ts index 3f5fad49..bb202b04 100644 --- a/src/cli/update-check.ts +++ b/src/cli/update-check.ts @@ -1,6 +1,7 @@ import { mkdir, readFile, writeFile } from "node:fs/promises"; import { homedir } from "node:os"; import { dirname, join } from "node:path"; +import { isObservationInvocation } from "../effects/read-policy.js"; const DEFAULT_TTL_MS = 24 * 60 * 60 * 1000; const DEFAULT_TIMEOUT_MS = 300; @@ -52,6 +53,7 @@ export async function maybeCheckForUpdate(options: UpdateCheckOptions): Promise< } function isDisabled(options: UpdateCheckOptions): boolean { + if (isObservationInvocation(options.argv ?? process.argv)) return true; const env = options.env ?? process.env; if (env.AGENTWHEEL_NO_UPDATE_CHECK === "1" || env.AGENTWHEEL_NO_UPDATE_CHECK === "true") return true; if (env.CI) return true; diff --git a/src/effects/read-policy.ts b/src/effects/read-policy.ts new file mode 100644 index 00000000..f1c69f36 --- /dev/null +++ b/src/effects/read-policy.ts @@ -0,0 +1,23 @@ +import { AsyncLocalStorage } from "node:async_hooks"; + +export type ReadEffectPolicy = "observe" | "refresh"; +const contexts = new AsyncLocalStorage(); + +export function readEffectPolicy(options: { refresh?: boolean; offline?: boolean }): ReadEffectPolicy { + return options.refresh === true && options.offline !== true ? "refresh" : "observe"; +} + +export function withReadEffects(options: { refresh?: boolean; offline?: boolean }, action: () => T): T { + return contexts.run(readEffectPolicy(options), action); +} + +export function isObservation(): boolean { return contexts.getStore() === "observe"; } + +export function requireRefreshEffect(effect: string): void { + if (isObservation()) throw new Error(`Observation refuses ${effect}. Use an explicit --refresh to update caches.`); +} + +// Conservative early suppression of the optional update notifier. Actual admission is scoped by the parsed action. +export function isObservationInvocation(argv: string[]): boolean { + return argv.includes("status") && readEffectPolicy({ refresh: argv.includes("--refresh"), offline: argv.includes("--offline") }) === "observe"; +} diff --git a/src/profile/members.ts b/src/profile/members.ts index 23b9044d..7d94442f 100644 --- a/src/profile/members.ts +++ b/src/profile/members.ts @@ -3,6 +3,7 @@ import { readFile } from "node:fs/promises"; import { join, resolve } from "node:path"; import { promisify } from "node:util"; import { z } from "zod"; +import { isObservation } from "../effects/read-policy.js"; import type { WorkspaceProfileMember } from "../model/workspace.js"; import { statusReportSchema, @@ -57,9 +58,9 @@ async function collectMember( const ageMs = cached ? Date.now() - new Date(cached.checkedAt).getTime() : Number.POSITIVE_INFINITY; const fresh = ageMs <= ttlSeconds * 1000; - if (options.offline || (cached && fresh && !options.refresh)) { + if (isObservation() || options.offline || (cached && fresh && !options.refresh)) { if (!cached) { - return memberFailure(member, "STALE", "No cached member status is available offline."); + return memberFailure(member, "STALE", "No cached member status is available; use --refresh explicitly."); } return memberFromReport(member, cached.report, { checkedAt: cached.checkedAt, diff --git a/src/registry/client.ts b/src/registry/client.ts index cccf3bf6..d75241a3 100644 --- a/src/registry/client.ts +++ b/src/registry/client.ts @@ -2,6 +2,7 @@ import { readFile, rm, stat } from "node:fs/promises"; import { homedir } from "node:os"; import { dirname, join, resolve } from "node:path"; import { fileURLToPath } from "node:url"; +import { isObservation } from "../effects/read-policy.js"; import { registryCacheSchema, registryIndexSchema, type RegistryCache, type RegistryEntry } from "../model/registry.js"; import { normalizeArtifactSelectors } from "../model/selection.js"; import { readMergedWorkspaceConfig } from "../model/workspace.js"; @@ -43,14 +44,16 @@ export class RegistryClient { } async getIndex(options: { refresh?: boolean } = {}): Promise { + const offline = this.options.offline || isObservation(); + const refresh = options.refresh && !isObservation(); const sources = await this.getSources(); const ttlMs = await this.getTtlMs(); const cached = await this.readCache(); - if (!options.refresh && cached && sameSources(cached.sources, sources) && (this.options.offline || !this.isExpired(cached, ttlMs))) { + if (!refresh && cached && sameSources(cached.sources, sources) && (offline || !this.isExpired(cached, ttlMs))) { this.warnCompatibility(cached.entries); return { entries: cached.entries, sources: cached.sources, fetchedAt: cached.fetchedAt, fromCache: true }; } - if (this.options.offline) { + if (offline) { const reason = cached ? "registry sources changed" : "registry cache is missing"; const label = this.options.offlineLabel ?? "Offline"; const hint = this.options.offlineHint ?? `Run without ${label === "Offline" ? "--offline" : "--frozen-lock"} first.`; diff --git a/src/resolve/graph.ts b/src/resolve/graph.ts index bbfc7e4b..d5f5ae1f 100644 --- a/src/resolve/graph.ts +++ b/src/resolve/graph.ts @@ -1,6 +1,7 @@ import { createHash } from "node:crypto"; import { readdir, readFile, stat } from "node:fs/promises"; import { basename, extname, join } from "node:path"; +import { isObservation } from "../effects/read-policy.js"; import { extractOpenPackIncludeSelectors, parseOpenPackIncludeSelector } from "../compose/markdown.js"; import type { Artifact, PackageItemRequire, PackageItemSuggest } from "../model/artifact.js"; import { artifactTypeSchema } from "../model/artifact.js"; @@ -1005,6 +1006,9 @@ async function fetchPackage( if (existing) return existing; const promise = (async () => { + if (isObservation() && !["local", "git"].includes(normalized.driver)) { + throw new Error(`Observation cannot qualify source driver '${normalized.driver}'; use --refresh explicitly.`); + } const driver = getSourceDriver(normalized.driver); const resolved = await driver.resolve(normalized.source, { cacheRoot: options.cacheRoot ?? join(options.workspaceRoot, ".agentwheel", "cache"), diff --git a/src/source/cache.ts b/src/source/cache.ts index cf51ce95..96a8760d 100644 --- a/src/source/cache.ts +++ b/src/source/cache.ts @@ -1,6 +1,7 @@ import { randomUUID } from "node:crypto"; import { mkdir, readdir, readFile, rm, stat, writeFile } from "node:fs/promises"; import { dirname, join } from "node:path"; +import { requireRefreshEffect } from "../effects/read-policy.js"; import { isIgnoredGeneratedEntry, pathExists } from "../utils/fs.js"; const snapshotNamePattern = /^(.*)-([0-9a-f]{12})$/i; @@ -30,6 +31,7 @@ interface SnapshotEntry { * The mutable checkout itself is never removed. */ export async function pruneGitCache(cacheRoot: string, options: GitCachePruneOptions = {}): Promise { + requireRefreshEffect("cache pruning"); if (!(await pathExists(cacheRoot))) return { removedPaths: [], retainedPaths: [] }; if (options.maintenanceLockHeld) return pruneGitCacheUnlocked(cacheRoot, options); return withGitCacheMaintenanceLock( @@ -44,6 +46,7 @@ export async function withGitCacheMaintenanceLock( timeoutMs: number, fn: () => Promise, ): Promise { + requireRefreshEffect("persistent cache maintenance locks"); const lockPath = join(cacheRoot, ".maintenance.lock"); await mkdir(cacheRoot, { recursive: true }); const started = Date.now(); @@ -73,6 +76,7 @@ export async function withGitCacheMaintenanceLock( } export async function createGitSnapshotLease(snapshotPath: string): Promise { + requireRefreshEffect("persistent snapshot leases"); const leasePath = `${snapshotPath}${leaseMarker}${process.pid}-${randomUUID()}`; await writeFile(leasePath, JSON.stringify({ pid: process.pid, createdAt: new Date().toISOString() }), { encoding: "utf8", diff --git a/src/source/git.ts b/src/source/git.ts index fdfe3011..ee047775 100644 --- a/src/source/git.ts +++ b/src/source/git.ts @@ -3,6 +3,7 @@ import { mkdir, rename, rm, stat } from "node:fs/promises"; import { homedir } from "node:os"; import { basename, dirname, join, resolve, sep } from "node:path"; import { promisify } from "node:util"; +import { isObservation } from "../effects/read-policy.js"; import { readPackageManifest } from "../model/package.js"; import { hashPath, @@ -43,6 +44,10 @@ export class GitSourceDriver implements SourceDriver { } async fetch(resolved: ResolvedSource): Promise { + if (isObservation()) return this.observe(resolved); + if (await pathExists(resolved.resolvedPath) && !(await pathExists(join(resolved.resolvedPath, ".git")))) { + throw incompleteCheckoutError(resolved.resolvedPath); + } return withFilesystemLock(`${resolved.resolvedPath}.lock`, resolved.cacheLockTimeoutMs ?? 30_000, async () => { const parsed = parseGitSource(resolved.source); const cacheRoot = dirname(resolved.resolvedPath); @@ -53,7 +58,9 @@ export class GitSourceDriver implements SourceDriver { if (resolved.frozenLock) { throw new Error(`Frozen lock requires cached git checkout at ${resolved.resolvedPath}`); } - await rm(resolved.resolvedPath, { recursive: true, force: true }); + if (await pathExists(resolved.resolvedPath)) { + throw incompleteCheckoutError(resolved.resolvedPath); + } await git([...(await gitAuthArguments(parsed.url)), "clone", "--no-checkout", parsed.url, resolved.resolvedPath]); await assertOwnedByCurrentUser(resolved.resolvedPath); } else if (!resolved.frozenLock) { @@ -103,6 +110,24 @@ export class GitSourceDriver implements SourceDriver { }, "git cache"); } + private async observe(resolved: ResolvedSource): Promise { + if (!(await pathExists(join(resolved.resolvedPath, ".git")))) { + if (await pathExists(resolved.resolvedPath)) throw incompleteCheckoutError(resolved.resolvedPath); + throw new Error(`Observation requires a cached Git checkout at ${JSON.stringify(resolved.resolvedPath)}; rerun the same scoped command with --refresh to create an absent cache.`); + } + const parsed = parseGitSource(resolved.source); + const commit = await resolveCommit(resolved.resolvedPath, resolved.requestedRef ?? parsed.ref ?? "HEAD"); + const snapshot = snapshotPathFor(resolved.resolvedPath, commit); + const packageRoot = parsed.subpath ? join(snapshot, parsed.subpath) : snapshot; + if (!(await isDirectory(packageRoot))) { + throw new Error("Observation requires an existing cached Git snapshot; use --refresh to materialize it."); + } + const manifest = await readPackageManifest(packageRoot); + return { ...resolved, resolvedPath: packageRoot, resolvedCommit: commit, + packageName: manifest?.name, packageVersion: manifest?.version, + sourceHash: await hashPath(packageRoot), cacheLeasePath: undefined }; + } + async list(resolved: ResolvedSource) { return this.local.list({ ...resolved, driver: "local" }); } @@ -139,7 +164,7 @@ function cachePathFor(url: string, cacheRoot?: string): string { } async function git(args: string[], env?: NodeJS.ProcessEnv) { - return execFileAsync("git", args, { env, maxBuffer: 1024 * 1024 * 10 }); + return execFileAsync("git", args, { env: isObservation() ? { ...process.env, ...env, GIT_OPTIONAL_LOCKS: "0" } : env, maxBuffer: 1024 * 1024 * 10 }); } async function resolveCommit(checkoutPath: string, ref: string): Promise { @@ -160,8 +185,21 @@ async function resolveCommit(checkoutPath: string, ref: string): Promise throw lastError; } +function incompleteCheckoutError(checkoutPath: string): Error { + return new Error( + `Incomplete cached checkout preserved at ${JSON.stringify(checkoutPath)}. ` + + `Manual repair: move this exact directory to a new backup location outside the cache directory ${JSON.stringify(dirname(checkoutPath))}; do not overwrite an existing backup. ` + + "Confirm the original directory is absent, then rerun the same scoped command with --refresh to clone it again. " + + "Agentwheel does not move or remove the directory automatically.", + ); +} + +function snapshotPathFor(checkoutPath: string, commit: string): string { + return join(dirname(checkoutPath), `${basename(checkoutPath)}-${commit.slice(0, 12)}`); +} + async function snapshotCommit(checkoutPath: string, commit: string): Promise<{ path: string; created: boolean }> { - const snapshotPath = join(dirname(checkoutPath), `${basename(checkoutPath)}-${commit.slice(0, 12)}`); + const snapshotPath = snapshotPathFor(checkoutPath, commit); if (await pathExists(snapshotPath)) return { path: snapshotPath, created: false }; const tempPath = join(dirname(checkoutPath), `${basename(snapshotPath)}.tmp-${process.pid}-${Date.now()}`); const tempIndexPath = `${tempPath}.index`; diff --git a/src/status/repository.ts b/src/status/repository.ts index 32373506..3492a606 100644 --- a/src/status/repository.ts +++ b/src/status/repository.ts @@ -19,7 +19,7 @@ export async function collectRepositoryStatus(workspaceRoot: string): Promise { it("reports correlated released state before target identity migration", async () => { const state = await releasedTargetStateFixture(); - - const status = JSON.parse((await runCli([ + const before = await snapshot(state.workspace); + const result = await runCli([ "status", "--adapter", "codex", "--installation-type", "local", "--target-root", state.workspace, "--json", - ])).stdout); + ]).catch((error: { code: number; stdout: string; stderr: string }) => { + // Observation reports unavailable version metadata without populating a cache. + expect(error.code, error.stdout + error.stderr).toBe(1); + return error; + }); + const status = JSON.parse(result.stdout); + expect(status.targets[0].packages[0].error).toMatch(/No cached version index.*--refresh/); + expect(diffEffects(before, await snapshot(state.workspace))).toEqual([]); expect(status.targets[0]).toMatchObject({ manifestEntryCount: 1, diff --git a/test/command-matrix.test.ts b/test/command-matrix.test.ts new file mode 100644 index 00000000..8f96d2aa --- /dev/null +++ b/test/command-matrix.test.ts @@ -0,0 +1,56 @@ +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join, relative } from "node:path"; +import { spawnSync } from "node:child_process"; +import { expect, it, vi } from "vitest"; +// @ts-expect-error The CLI support script is native ESM JavaScript. +import { reconcile, counts, runTests, matrixDestination, parserCommands } from "../scripts/test-command-matrix.mjs"; + +vi.mock("node:child_process", async importOriginal => ({ + ...await importOriginal(), spawnSync: vi.fn(), +})); + +it("reconciles shared command attribution without double-counting unique runner IDs", () => { + const registered = [{ id: "a", file: "case.ts", name: "parameter 1" }, { id: "b", file: "case.ts", name: "parameter 2" }]; + const results = registered.map(test => ({ ...test, status: "passed" })); + const matrix = reconcile(["one", "two", "zero"], registered, results, + [{ file: "case.ts", name: "parameter 1", commands: ["one", "two"], evidence: "shared fixture" }]); + expect(matrix.uniqueAttributed).toBe(1); + expect(matrix.rows.map((row: { registered: number }) => row.registered)).toEqual([1,1,0]); + expect(matrix.unattributed.map((test: { id: string }) => test.id)).toEqual(["b"]); + expect(() => reconcile(["one"], registered, [results[0]], [])).toThrow(/reconcile/); + expect(() => reconcile(["one"], registered, results, [{ file: "case.ts", name: "missing", commands: ["one"], evidence: "fixture" }])).toThrow(/exactly one/); + expect(() => reconcile(["one"], registered, results, [{ file: "case.ts", name: "parameter 1", commands: ["unknown"], evidence: "fixture" }])).toThrow(/Unknown/); + expect(counts([{ status: "passed" }, { status: "failed" }, { status: "skipped" }, { status: "pending" }])).toEqual({ registered: 4, executed: 2, pass: 1, fail: 1, skipped: 1, pending: 1 }); +}); + +it("rejects mappings without a commands array with an explicit validation error", () => { + const registered = [{ id: "a", file: "case.ts", name: "case" }]; + for (const commands of [undefined, null, "one", []]) { + expect(() => reconcile(["one"], registered, registered, + [{ file: "case.ts", name: "case", evidence: "fixture", commands }])).toThrow(/Invalid mapping/); + } +}); + +it("streams default reporter output and runner errors to CI", () => { + runTests("runner.json", "vitest.json"); + expect(vi.mocked(spawnSync)).toHaveBeenCalledWith("pnpm", expect.arrayContaining(["test", "--reporter=default"]), + expect.objectContaining({ stdio: "inherit" })); +}); + +it("keeps the default report outside the checkout in a temporary directory", async () => { + const output = await matrixDestination(); + try { + expect(relative(tmpdir(), output)).toMatch(/^command-matrix-output-[^/]+\/matrix\.json$/); + expect(await matrixDestination("custom.json")).toBe(join(process.cwd(), "custom.json")); + } finally { await rm(dirname(output), { recursive: true, force: true }); } +}); + +it("loads the Commander tree from a scratch directory outside the checkout", async () => { + const scratch = await mkdtemp(join(tmpdir(), "command-matrix-parser-test-")); + try { + const commands = await parserCommands(scratch); + expect(commands).toEqual(expect.arrayContaining(["agentwheel", "cache prune", "registry publish", "doctor"])); + expect(new Set(commands).size).toBe(commands.length); + } finally { await rm(scratch, { recursive: true, force: true }); } +}); diff --git a/test/effects-harness.test.ts b/test/effects-harness.test.ts new file mode 100644 index 00000000..d8f8582a --- /dev/null +++ b/test/effects-harness.test.ts @@ -0,0 +1,35 @@ +import { execFileSync } from "node:child_process"; +import { chmod, mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it } from "vitest"; +import { diffEffects, snapshot } from "./helpers/effects.js"; + +it("detects bytes, modes, symlinks, claims, index and refs without following links", async () => { + const root = await mkdtemp(join(tmpdir(), "effects-harness-")); + try { + const git = (...args: string[]) => execFileSync("git", ["-C", root, ...args]); + git("init", "-b", "main"); + git("config", "user.name", "Synthetic"); + git("config", "user.email", "synthetic@example.invalid"); + await writeFile(join(root, "file"), Buffer.from([0,255,1])); + await writeFile(join(root, "manifest.json"), JSON.stringify({ entries: [{ owner: "alpha" }] })); + await symlink("file", join(root,"link")); + git("add", "file"); git("commit", "-m", "fixture"); + const policy = { manifests: ["manifest.json"], git: true }; + const before = await snapshot(root, policy); + expect(diffEffects(before, await snapshot(root,policy))).toEqual([]); + await writeFile(join(root,"file"), Buffer.from([0,255,2])); + await chmod(join(root,"file"), 0o755); + await rm(join(root,"link")); await symlink("missing",join(root,"link")); + await writeFile(join(root,"manifest.json"), JSON.stringify({ entries: [{ owner: "beta" }] })); + git("add", "file"); git("branch", "other"); + const changes = diffEffects(before, await snapshot(root,policy)); + expect(changes).toEqual(expect.arrayContaining(["tree:file", "tree:link", "claims:manifest.json", "git:index", "git:refs"])); + await expect(snapshot(root, { exclude: { "**": "invalid" } })).rejects.toThrow(/exact/); + await expect(snapshot(root, { exclude: { "file": "" } })).rejects.toThrow(/reason/); + const excluded = { exclude: { "file": "This fixture explicitly ignores the mutable file payload." } }; + expect((await snapshot(root,excluded)).tree.file).toBeUndefined(); + await expect(snapshot(root, { manifests: ["manifest.json"], exclude: { "manifest.json": "invalid claim exclusion" } })).rejects.toThrow(/claims/); + } finally { await rm(root, { recursive: true, force: true }); } +}); diff --git a/test/helpers/effects.ts b/test/helpers/effects.ts new file mode 100644 index 00000000..2140d645 --- /dev/null +++ b/test/helpers/effects.ts @@ -0,0 +1,58 @@ +import { lstat, readdir, readFile, readlink } from "node:fs/promises"; +import { isAbsolute, join, resolve } from "node:path"; +import { execFileSync } from "node:child_process"; + +export interface EffectSnapshot { + tree: Record; + claims: Record; + git?: { index: string; refs: string; head: string; indexBytes: string }; +} +export interface EffectPolicy { + // Exact relative paths only, each with a fixture-specific reason. No globs or prefixes. + exclude?: Record; + manifests?: string[]; + git?: boolean; +} + +export async function snapshot(root: string, policy: EffectPolicy = {}): Promise { + for (const [path, reason] of Object.entries(policy.exclude ?? {})) { + if (!reason.trim() || path.startsWith("/") || path.split("/").some(p => p === ".." || p === "." || !p) + || /[*?\[\]]/.test(path)) throw new Error("Exclusions require an exact relative path and reason"); + } + const result: EffectSnapshot = { tree: {}, claims: {} }; + async function visit(relative: string): Promise { + const path = join(root, relative); + const stat = await lstat(path); + if (stat.isDirectory()) { + result.tree[relative || "."] = { kind: "directory", mode: stat.mode & 0o7777 }; + for (const name of (await readdir(path)).sort()) await visit(relative ? `${relative}/${name}` : name); + } else if (!Object.hasOwn(policy.exclude ?? {}, relative)) { + result.tree[relative] = stat.isSymbolicLink() + ? { kind: "symlink", mode: stat.mode & 0o7777, target: await readlink(path) } + : { kind: "file", mode: stat.mode & 0o7777, bytes: (await readFile(path)).toString("base64") }; + } else if ((policy.manifests ?? []).includes(relative)) { + throw new Error("Manifest claims cannot be excluded"); + } + } + await visit(""); + for (const path of policy.manifests ?? []) result.claims[path] = JSON.parse(await readFile(join(root, path), "utf8")); + if (policy.git) { + const git = (...args: string[]) => execFileSync("git", ["-C", root, ...args], { encoding: "utf8" }); + const indexPath = git("rev-parse", "--git-path", "index").trim(); + const indexBytes = (await readFile(isAbsolute(indexPath) ? indexPath : resolve(root,indexPath))).toString("base64"); + result.git = { indexBytes, index: git("ls-files", "--stage", "-z"), refs: git("show-ref", "--head"), head: git("symbolic-ref", "-q", "HEAD") }; + } + return result; +} + +export function diffEffects(before: EffectSnapshot, after: EffectSnapshot): string[] { + const changes: string[] = []; + for (const section of ["tree", "claims", "git"] as const) { + const left = before[section] ?? {}; + const right = after[section] ?? {}; + for (const key of [...new Set([...Object.keys(left), ...Object.keys(right)])].sort()) { + if (JSON.stringify(left[key as keyof typeof left]) !== JSON.stringify(right[key as keyof typeof right])) changes.push(`${section}:${key}`); + } + } + return changes; +} diff --git a/test/helpers/read-effects-preload.cjs b/test/helpers/read-effects-preload.cjs new file mode 100644 index 00000000..54cdf641 --- /dev/null +++ b/test/helpers/read-effects-preload.cjs @@ -0,0 +1,22 @@ +// Fixture-only observer. Evidence is the single explicitly excluded trace file. +const fs = require("node:fs"); +const child = require("node:child_process"); +const util = require("node:util"); +const { syncBuiltinESMExports } = require("node:module"); +function record(kind, name, args) { + fs.appendFileSync(process.env.SYNTHETIC_EFFECT_TRACE, JSON.stringify({ kind, name, args }) + "\n"); +} +for (const name of ["execFile", "execFileSync", "spawn", "spawnSync", "exec", "execSync"]) { + const original = child[name]; + const wrapped = function(...args) { record("process", name, args.slice(0,2)); return original.apply(this,args); }; + if (original[util.promisify.custom]) wrapped[util.promisify.custom] = function(...args) { + record("process", name, args.slice(0,2)); return original[util.promisify.custom].apply(this,args); + }; + child[name] = wrapped; +} +for (const [moduleName, names] of [["node:http", ["request", "get"]], ["node:https", ["request", "get"]], ["node:net", ["connect", "createConnection"]]]) { + const module = require(moduleName); + for (const name of names) module[name] = function() { record("network", `${moduleName}.${name}`, []); throw new Error("Synthetic fixture forbids network"); }; +} +globalThis.fetch = async function() { record("network", "fetch", []); throw new Error("Synthetic fixture forbids network"); }; +syncBuiltinESMExports(); diff --git a/test/helpers/read-effects.ts b/test/helpers/read-effects.ts new file mode 100644 index 00000000..83a84af3 --- /dev/null +++ b/test/helpers/read-effects.ts @@ -0,0 +1,52 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import { mkdir, mkdtemp, readFile, rm, utimes, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { GitSourceDriver } from "../../src/source/git.js"; +import { releaseGitSnapshotLease } from "../../src/source/cache.js"; +import { diffEffects, snapshot } from "./effects.js"; + +export async function readFixture(warm: boolean) { + const root = await mkdtemp(join(tmpdir(), "synthetic-read-")); + const home = join(root,"home"); const workspace = join(root,"workspace"); + const temp = join(root,"temp"); const source = join(root,"source"); + const cacheRoot = join(workspace,".agentwheel/cache"); + for (const path of [home, workspace, temp, join(source,"skills/synthetic"), join(workspace,".agentwheel")]) await mkdir(path, { recursive: true }); + const git = (dir: string, ...args: string[]) => execFileSync("git", ["-C",dir,...args], { encoding: "utf8", env: { ...process.env, GIT_CONFIG_GLOBAL: "/dev/null" } }); + for (const dir of [workspace, source]) { + git(dir,"init","-b","main"); git(dir,"config","user.name","Synthetic"); git(dir,"config","user.email","synthetic@example.invalid"); + } + await writeFile(join(workspace,"sentinel"),"user bytes"); git(workspace,"add","sentinel"); git(workspace,"commit","-m","fixture"); + await writeFile(join(source,"skills/synthetic/SKILL.md"), "---\nname: synthetic\ndescription: Synthetic skill.\n---\n# Synthetic\n"); + await writeFile(join(source,"openpack.json"), JSON.stringify({ schemaVersion: 2, name: "fixture/synthetic", version: "1.0.0", provides: [{ type: "skills", path: "skills" }] })); + git(source,"add","openpack.json","skills"); git(source,"commit","-m","fixture"); + const sourceUrl = `git:${source}#main`; + await writeFile(join(workspace,".agentwheel/config.json"), JSON.stringify({ schemaVersion: 1, bootstrapSkills: false, + packages: [{ name: "synthetic", source: sourceUrl, driver: "git", mode: "pinned", requestedRef: "main", adapter: "codex", installationType: "local" }], registry: {}, profiles: {}, agents: {} })); + const driver = new GitSourceDriver(); + const resolved = await driver.resolve(sourceUrl,{ cacheRoot }); + if (warm) { + const fetched = await driver.fetch(resolved); await releaseGitSnapshotLease(fetched.cacheLeasePath); + for (const prefix of ["111111111111","222222222222","333333333333","444444444444"]) { + const path = `${resolved.resolvedPath}-${prefix}`; + await mkdir(path); await writeFile(join(path,"sentinel"),"retained cache evidence"); await utimes(path,1,1); + } + } + const trace = join(root,"trace.jsonl"); await writeFile(trace,""); + const policy = { exclude: { "trace.jsonl": "Only the fixture observer evidence is excluded; command state is compared exactly." } }; + return { + root, workspace, home, temp, cacheRoot, resolved, + async probe(extra: string[] = [], entry = join(process.cwd(),"dist/index.js"), env: NodeJS.ProcessEnv = {}) { + const before = await snapshot(root,policy); + const result = spawnSync(process.execPath, [entry,"status","--installation-type","local","--adapter","codex","--target-root",workspace,"--json",...extra], { + cwd: workspace, encoding: "utf8", timeout: 30_000, + env: { ...process.env, HOME: home, XDG_CONFIG_HOME: join(home,".config"), XDG_CACHE_HOME: join(home,".cache"), TMPDIR: temp, + AGENTWHEEL_NO_UPDATE_CHECK: "1", GIT_CONFIG_GLOBAL: "/dev/null", + NODE_OPTIONS: `--require=${resolve("test/helpers/read-effects-preload.cjs")}`, SYNTHETIC_EFFECT_TRACE: trace, ...env }, + }); + const calls = (await readFile(trace,"utf8")).trim().split("\n").filter(Boolean).map(line => JSON.parse(line) as { kind: string; name: string; args: unknown[] }); + return { result, calls, changes: diffEffects(before,await snapshot(root,policy)) }; + }, + dispose: () => rm(root,{ recursive: true, force: true }), + }; +} diff --git a/test/invariants/i2-no-loss.test.ts b/test/invariants/i2-no-loss.test.ts new file mode 100644 index 00000000..7074a935 --- /dev/null +++ b/test/invariants/i2-no-loss.test.ts @@ -0,0 +1,68 @@ +import { mkdir, mkdtemp, readFile, rename, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, relative } from "node:path"; +import { expect, it } from "vitest"; +import { withReadEffects } from "../../src/effects/read-policy.js"; +import { GitSourceDriver } from "../../src/source/git.js"; +import { releaseGitSnapshotLease } from "../../src/source/cache.js"; +import { pathExists } from "../../src/utils/fs.js"; +import { readFixture } from "../helpers/read-effects.js"; +import { diffEffects, snapshot } from "../helpers/effects.js"; + +async function observeIncompleteCacheRead(observe = false) { + const root = await mkdtemp(join(tmpdir(), "invariant-i2-")); + try { + const driver = new GitSourceDriver(); + // Local nonexistent source: clone must fail without network access. + const resolved = await driver.resolve(`git:file://${join(root, "missing.git")}`, { cacheRoot: join(root, "cache") }); + await mkdir(resolved.resolvedPath, { recursive: true }); + await writeFile(join(resolved.resolvedPath, "sentinel"), "preserve me"); + const before = await snapshot(root); + const failure = await withReadEffects({ refresh: !observe }, () => driver.fetch(resolved)).then(() => undefined, error => error); + const cachePath = relative(root, resolved.resolvedPath); + return { failure, checkoutPath: resolved.resolvedPath, effects: diffEffects(before, await snapshot(root)), + removedPaths: [`tree:${cachePath}`, `tree:${cachePath}/sentinel`] }; + + } finally { await rm(root, { recursive: true, force: true }); } +} + +// Expected: a read refuses an incomplete checkout without deleting existing bytes. +it("read_missing_git_preserves_cache", async () => { + const observed = await observeIncompleteCacheRead(); + expect(observed.failure).toBeInstanceOf(Error); + expect(observed.effects).toEqual([]); +}); + +it("read_missing_git_preserves_cache_requires_explicit_repair", async () => { + const observed = await observeIncompleteCacheRead(); + expect(observed.failure).toBeInstanceOf(Error); + expect(String(observed.failure)).toMatch(/Incomplete cached checkout preserved/); + expect(observed.effects).toEqual([]); +}); + +it.each([false, true])("incomplete_cache_error_names_manual_move_observe_%s", async observe => { + const observed = await observeIncompleteCacheRead(observe); + expect(observed.failure).toBeInstanceOf(Error); + const message = String(observed.failure); + expect(message).toContain(JSON.stringify(observed.checkoutPath)); + expect(message).toMatch(/Manual repair: move this exact directory.*backup location outside the cache directory/); + expect(message).toMatch(/Confirm the original directory is absent, then rerun the same scoped command with --refresh/); + expect(observed.effects).toEqual([]); +}); + +it("manual_backup_allows_refresh_without_losing_incomplete_checkout_evidence", async () => { + const fixture = await readFixture(true); + try { + await rm(join(fixture.resolved.resolvedPath, ".git"), { recursive: true }); + await writeFile(join(fixture.resolved.resolvedPath, "sentinel"), "preserved evidence"); + const driver = new GitSourceDriver(); + await expect(withReadEffects({refresh:true}, () => driver.fetch(fixture.resolved))).rejects.toThrow(JSON.stringify(fixture.resolved.resolvedPath)); + const backup = join(fixture.root, "checkout-backup"); + // This is the explicit operator step, outside the cache, against a new destination. + await rename(fixture.resolved.resolvedPath, backup); + const refreshed = await withReadEffects({refresh:true}, () => driver.fetch(fixture.resolved)); + await releaseGitSnapshotLease(refreshed.cacheLeasePath); + expect(await readFile(join(backup, "sentinel"), "utf8")).toBe("preserved evidence"); + expect(await pathExists(join(fixture.resolved.resolvedPath, ".git"))).toBe(true); + } finally { await fixture.dispose(); } +}); diff --git a/test/invariants/i5-admission-parity.test.ts b/test/invariants/i5-admission-parity.test.ts new file mode 100644 index 00000000..f94853b8 --- /dev/null +++ b/test/invariants/i5-admission-parity.test.ts @@ -0,0 +1,51 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it } from "vitest"; +import { createCombinedInstallPlan, applyCombinedInstallPlan, readInstallManifest, type DesiredArtifact } from "../../src/install/index.js"; +import { targetMappingSchema, type AdapterConfig } from "../../src/model/adapter.js"; +import { hashPath } from "../../src/utils/fs.js"; +import { diffEffects, snapshot } from "../helpers/effects.js"; + +async function observeStaleMergeAdmission() { + const root = await mkdtemp(join(tmpdir(), "invariant-i5-")); + try { + const sourcePath = join(root, "source.json"); + await writeFile(sourcePath, JSON.stringify({ managed: { value: "current" } })); + const adapter: AdapterConfig = { name: "synthetic", targets: { settings: { + local: targetMappingSchema.parse({ enabled: true, dest: "config.json", merge: "json-deep" }), + } } }; + const artifact: DesiredArtifact = { type: "settings", name: "synthetic", sourcePath, stagedPath: sourcePath, + relativePath: "source.json", kind: "file", hash: await hashPath(sourcePath), channel: "managed", meta: { dependencyRole: "root", owners: ["synthetic"], logicalSelector: "settings/synthetic" } }; + await applyCombinedInstallPlan(await createCombinedInstallPlan([artifact], adapter, root)); + const manifest = (await readInstallManifest(root, adapter.name))!; + manifest.entries[0]!.mergeRemoval = { managed: { value: "stale" } }; + // Source hash is unchanged; user-owned fields must survive both admission paths. + await writeFile(join(root, "config.json"), JSON.stringify({ managed: { value: "current" }, user: { keep: true } })); + const before = await snapshot(root); + const preview = await createCombinedInstallPlan([artifact], adapter, root, manifest); + expect(diffEffects(before, await snapshot(root))).toEqual([]); + const failure = await applyCombinedInstallPlan(preview).then(() => undefined, error => error); + return { preview, failure, effects: diffEffects(before, await snapshot(root)) }; + } finally { await rm(root, { recursive: true, force: true }); } +} + +// Expected: preview blocks the same stale contribution that apply rejects. +it.fails("stale_merge_metadata_rejected_in_all_paths", async () => { + const { preview, failure, effects } = await observeStaleMergeAdmission(); + expect(failure).toBeInstanceOf(Error); + expect(preview.hasBlockingChanges).toBe(true); + expect(preview.operations).toEqual(expect.arrayContaining([expect.objectContaining({ + action: "conflict", blockedReason: expect.stringMatching(/contribution|merge/i), + })])); + expect(effects).toEqual([]); +}); + +it("stale_merge_metadata_rejected_in_all_paths_current_defect_skips_preview_but_apply_refuses", async () => { + const { preview, failure } = await observeStaleMergeAdmission(); + expect(preview.hasBlockingChanges).toBe(false); + expect(preview.operations).toMatchObject([{ action: "skip" }]); + expect(preview.operations).toHaveLength(1); + expect(failure).toBeInstanceOf(Error); + expect(String(failure)).toMatch(/exact merge contribution/); +}); diff --git a/test/invariants/i7-effects-containment.test.ts b/test/invariants/i7-effects-containment.test.ts new file mode 100644 index 00000000..1fedca0a --- /dev/null +++ b/test/invariants/i7-effects-containment.test.ts @@ -0,0 +1,112 @@ +import { execFileSync } from "node:child_process"; +import { mkdir, rm, utimes, writeFile } from "node:fs/promises"; +import { join, resolve } from "node:path"; +import { expect, it } from "vitest"; +import { readFixture } from "../helpers/read-effects.js"; + +it.each([false,true])("read_effects_match_explicit_policy_status_warm_%s", async warm => { + const fixture = await readFixture(warm); + try { + const observed = await fixture.probe(); + expect(observed.result.error).toBeUndefined(); + expect(observed.result.stdout, observed.result.stderr).not.toBe(""); + const report = JSON.parse(observed.result.stdout); + expect(report.command).toBe("status"); + if (warm) expect(report.targets[0].pendingCount).toBeGreaterThan(0); + expect(observed.calls.filter(call => call.kind === "network")).toEqual([]); + expect.soft(observed.calls.filter(call => /\b(fetch|clone|ls-remote)\b/.test(JSON.stringify(call.args)))).toEqual([]); + expect.soft(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_never_evicts_snapshot", async () => { + const fixture = await readFixture(true); + try { + const observed = await fixture.probe(["--offline"]); + expect(observed.result.stdout, observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect.soft(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_missing_git_preserves_cache_status", async () => { + const fixture = await readFixture(true); + try { + await rm(join(fixture.resolved.resolvedPath,".git"), { recursive: true }); + await writeFile(join(fixture.resolved.resolvedPath,"sentinel"),"preserve incomplete checkout"); + const observed = await fixture.probe(); + expect(observed.result.stdout, observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect.soft(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_effects_match_explicit_policy_diagnostics_consumer", async () => { + const fixture = await readFixture(true); + try { + const consumer = join(fixture.root,"diagnostics.mjs"); + await writeFile(consumer, `import { spawnSync } from "node:child_process"; +const result = spawnSync(process.execPath, [process.env.SYNTHETIC_CLI,...process.argv.slice(2)], {stdio:"inherit"}); +process.exitCode = result.status ?? 1;`); + const observed = await fixture.probe([],consumer,{ SYNTHETIC_CLI: resolve("dist/index.js") }); + expect(observed.result.stdout,observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect(observed.calls.filter(call => call.kind === "network")).toEqual([]); + expect(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("unqualified_cli_does_not_build_on_read", async () => { + const fixture = await readFixture(false); + try { + const observed = await fixture.probe([],resolve("scripts/agentwheel-local.mjs"),{ AGENTWHEEL_REPO_DIR: fixture.root }); + expect(observed.result.status).toBe(127); + expect(observed.result.stderr).toMatch(/Build explicitly.*never builds implicitly/); + expect(observed.calls).toEqual([]); + expect(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_effects_match_explicit_policy_local_wrapper", async () => { + const fixture = await readFixture(true); + try { + const observed = await fixture.probe([],resolve("scripts/agentwheel-local.mjs"),{ AGENTWHEEL_REPO_DIR: process.cwd() }); + expect(observed.result.stdout,observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect(observed.calls.filter(call => call.kind === "network")).toEqual([]); + expect(observed.calls.filter(call => /\b(pnpm|flock|clone|fetch|ls-remote)\b/.test(JSON.stringify(call.args)))).toEqual([]); + expect(observed.changes).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_effects_match_explicit_policy_explicit_refresh", async () => { + const fixture = await readFixture(false); + try { + const observed = await fixture.probe(["--refresh"]); + expect(observed.result.stdout,observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect(observed.changes.length).toBeGreaterThan(0); + expect(observed.changes.every(path => path.startsWith("tree:workspace/.agentwheel/cache"))).toBe(true); + expect(observed.calls.filter(call => call.kind === "network")).toEqual([]); + } finally { await fixture.dispose(); } +}, 30_000); + +it("read_never_evicts_snapshot_warm_cache", async () => { + const fixture = await readFixture(true); + try { + await mkdir(join(fixture.cacheRoot,"synthetic-checkout/.git"),{recursive:true}); + for (const suffix of ["aaaaaaaaaaaa","bbbbbbbbbbbb","cccccccccccc","dddddddddddd"]) { + const dir = join(fixture.cacheRoot,`synthetic-checkout-${suffix}`); + await mkdir(dir); await writeFile(join(dir,"sentinel"),"recovery evidence"); + } + // A newer source commit makes the old reader materialize a snapshot and run global pruning. + const source = join(fixture.root,"source"); + await writeFile(join(source,"revision"),"new revision"); + execFileSync("git",["-C",source,"add","revision"]); + execFileSync("git",["-C",source,"commit","-m","synthetic revision"]); + const observed = await fixture.probe([],process.env.SYNTHETIC_READ_CLI); + expect(observed.result.stdout, observed.result.stderr).not.toBe(""); + expect(JSON.parse(observed.result.stdout).command).toBe("status"); + expect(observed.changes).toEqual([]); + } finally {await fixture.dispose();} +},30_000); diff --git a/test/read-effect-policy.test.ts b/test/read-effect-policy.test.ts new file mode 100644 index 00000000..bd63b371 --- /dev/null +++ b/test/read-effect-policy.test.ts @@ -0,0 +1,70 @@ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { expect, it } from "vitest"; +import { withReadEffects, isObservation, readEffectPolicy, isObservationInvocation } from "../src/effects/read-policy.js"; +import { createGitSnapshotLease, pruneGitCache } from "../src/source/cache.js"; +import { maybeCheckForUpdate } from "../src/cli/update-check.js"; +import { collectCompositeMembers } from "../src/profile/members.js"; +import { resolveAdapter } from "../src/adapters/resolve.js"; +import { diffEffects, snapshot } from "./helpers/effects.js"; + +it("preserves context across asynchronous work and restores the caller scope", async () => { + expect(isObservation()).toBe(false); + await withReadEffects({}, async () => { + await Promise.resolve(); expect(isObservation()).toBe(true); + await withReadEffects({refresh:true}, async () => { expect(isObservation()).toBe(false); }); + expect(isObservation()).toBe(true); + }); + expect(isObservation()).toBe(false); + expect(readEffectPolicy({refresh:true,offline:true})).toBe("observe"); +}); + +it("refuses persistent lease and prune helpers before touching an observation tree", async () => { + const root = await mkdtemp(join(tmpdir(),"synthetic-policy-")); + try { + const before = await snapshot(root); + await withReadEffects({}, async () => { + await expect(createGitSnapshotLease(join(root,"snapshot"))).rejects.toThrow(/Observation refuses/); + await expect(pruneGitCache(root)).rejects.toThrow(/Observation refuses/); + }); + expect(diffEffects(before,await snapshot(root))).toEqual([]); + } finally { await rm(root,{recursive:true,force:true}); } +}); + +it("does not fetch or write update notices on an interactive status read", async () => { + let calls=0; + await maybeCheckForUpdate({currentVersion:"1.0.0",argv:["node","cli","status"],env:{},isTTY:true, + fetchImpl: async () => {calls++; throw new Error("unexpected network");}}); + expect(calls).toBe(0); +}); + +it("does not invoke or cache an uncached composite member during observation", async () => { + const root = await mkdtemp(join(tmpdir(),"synthetic-members-")); + try { + const before = await snapshot(root); + const members = await withReadEffects({}, () => collectCompositeMembers({cliVersion:"1.0.0",workspaceRoot:root, + profileName:"synthetic",profileTtlSeconds:1,members:[{id:"leaf",workspace:root,profile:"synthetic",transport:"local"}]})); + expect(members[0]?.error).toMatch(/No cached member status/); + expect(diffEffects(before,await snapshot(root))).toEqual([]); + } finally {await rm(root,{recursive:true,force:true});} +}); + +it("refuses unknown adapter effects before evaluating the module", async () => { + const root = await mkdtemp(join(tmpdir(),"synthetic-adapter-")); + try { + const module = join(root,"adapter.mjs"); + await writeFile(module, `import {writeFileSync} from "node:fs";writeFileSync(${JSON.stringify(join(root,"side-effect"))},"unexpected");export default {};`); + const before = await snapshot(root); + await withReadEffects({},async () => { + await expect(resolveAdapter({adapterModule:module,allowAdapterCode:true})).rejects.toThrow(/Observation refuses unqualified adapter/); + }); + expect(diffEffects(before,await snapshot(root))).toEqual([]); + } finally {await rm(root,{recursive:true,force:true});} +}); + +it("conservatively suppresses the notifier when status is an argument value", () => { + expect(isObservationInvocation(["node", "cli", "add", "status"])).toBe(true); + expect(isObservationInvocation(["node", "cli", "status", "--refresh"])).toBe(false); + expect(isObservationInvocation(["node", "cli", "status", "--refresh", "--offline"])).toBe(true); +});