diff --git a/README.md b/README.md index 58d2cb73..66242053 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ For a one-off local check, wrap the test command with `tdd run`: ```bash vizzly tdd run "pnpm test" --no-open -vizzly context build current --source local --agent +vizzly context build current --source local --json ``` That run writes review data under `.vizzly/` and prints a context command you @@ -96,38 +96,44 @@ vizzly run "pnpm test" --wait ### Inspect Builds And Diffs -Use `vizzly context` when you need build, comparison, screenshot, or review -queue data from the terminal. +Use `vizzly context` for human summaries and local `.vizzly` evidence. For cloud +reviews, agents should discover and call the public API through its schema: -This is useful for scripts, coding agents, and debugging loops. Instead of -making a pile of narrow API calls, ask for one focused bundle and get the -evidence in one place. +```bash +vizzly api schema --json +vizzly api schema sdk.listBuilds --json +``` + +The `context build --agent` and `context comparison --agent` formats are +deprecated and scheduled for removal in v0.38.0. Use the schema-driven API for +cloud evidence; use `vizzly context ... --json` for local evidence. ```bash -# Cloud context for a build or comparison +# Human-readable cloud build context vizzly context build abc123 --source cloud -vizzly context comparison def456 --source cloud --agent --json + +# Discover cloud evidence operations for an agent workflow +vizzly api schema sdk.getBuildContext --json +vizzly api schema getComparisonContext --json # Local workspace context from .vizzly/ vizzly context build current --source local -vizzly context build current --source local --agent +vizzly context build current --source local --json vizzly context screenshot build-detail-screenshots --source local --json vizzly context review-queue --source local --json ``` -`--json` is the durable automation path. `--agent` gives you the compact API -handoff for prompt assembly. Build handoffs contain up to 10 records; use the -exact next-page command returned in `suggested_commands` to continue safely. -That command carries the API's opaque `--cursor`. Add -`--include diffs` for raw diff diagnostics, or `--full` when you need the whole -payload. +Use `vizzly api schema` to inspect cloud operation inputs, output fields, and +pagination before making requests. The old compact `--agent` handoff remains +available during deprecation, but new cloud workflows should use the schema. +Use `--json` for machine-readable local context. Local context is read-only and file-backed. It reads your existing `.vizzly` workspace state from TDD runs, including screenshots, diffs, and saved hotspot or region metadata. -Cloud context is also read-only right now. That is intentional. Vizzly helps you -see and inspect visual changes, while people still decide what gets approved. +The `context` commands are read-only. The schema-discovered review API also +supports explicit decisions when a task asks for them. ## Capture Screenshots diff --git a/docs/json-output.md b/docs/json-output.md index 5702f0e0..35ad7168 100644 --- a/docs/json-output.md +++ b/docs/json-output.md @@ -71,7 +71,7 @@ vizzly run "pnpm test" --json "data": { "buildId": "abc123-def456", "status": "completed", - "contextCommand": "vizzly context build abc123-def456 --agent --json --source cloud", + "contextCommand": "vizzly api schema sdk.getBuildContext --json", "screenshotsCaptured": 15, "executionTimeMs": 4821, "git": { @@ -84,6 +84,10 @@ vizzly run "pnpm test" --json } ``` +For cloud runs, `contextCommand` starts schema discovery for the build-context +operation. Use the `buildId` in the response when following that operation's +path to inspect build evidence. + With `--wait`, includes comparison results: ```json @@ -106,7 +110,7 @@ With `--wait`, includes comparison results: "identical": 12 }, "visual_review": { "state": "pending" }, - "contextCommand": "vizzly context build abc123-def456 --agent --json --source cloud", + "contextCommand": "vizzly api schema sdk.getBuildContext --json", "exitCode": 1 } } @@ -147,7 +151,7 @@ vizzly tdd run "pnpm test" --json "new": 0 }, "reportPath": ".vizzly/report/index.html", - "contextCommand": "vizzly context build current --source local --agent --json" + "contextCommand": "vizzly context build current --source local --json" } } ``` @@ -252,9 +256,15 @@ vizzly tdd list --json ### `vizzly context` -Use `vizzly context` when you want one machine-friendly bundle instead of several narrow calls. -This is the best fit for automation, agents, and scripts that need approved baselines, visual -evidence, review state, comments, preview links, and diff metadata in one place. +> **Deprecation:** `--agent` on `context build` and `context comparison` remains +> supported through v0.37.x, with a warning, and is scheduled for removal in +> v0.38.0. New cloud agent workflows should use `vizzly api schema --json` and +> the discovered API operations. Use `vizzly context ... --json` for local +> evidence. The compact payload examples below document the legacy format. + +Use `vizzly context` for human summaries and local workspace data. The compact +agent format below is kept for compatibility during deprecation. For new cloud +agent workflows, discover the public API with `vizzly api schema --json`. Every context payload includes a `source` field. That tells you whether the bundle came from cloud data or your local `.vizzly` workspace. @@ -270,13 +280,13 @@ vizzly context build current --source local --json vizzly context build current --source local --agent ``` -Use `--json` for durable automation. Use `--agent --json` when you want the compact handoff that -agents should read first. The API chooses and orders up to 10 evidence records, then returns an -opaque cursor when more evidence is available. Follow the suggested next-page command or pass that -cursor to `--cursor`. Add `--include diffs` for raw Honeydiff diagnostics on the same page. -`--full` returns the complete build context payload unchanged. +In the legacy format, `--agent --json` selects a compact handoff. It remains +available through v0.37.x, but new cloud agent workflows should start with the +schema-discovered API. Legacy build handoffs contain up to 10 evidence records +and return an opaque cursor when more evidence is available. `--full` returns +the complete build context payload unchanged. -Compact agent JSON: +Legacy compact build JSON: ```json { @@ -455,11 +465,12 @@ vizzly context comparison cmp-1 --source cloud --agent --json --include diffs vizzly context comparison build-detail-screenshots --source local --json ``` -Raw JSON preserves the provider response. Add `--agent` to request the compact API shape. The focal -comparison stays in its API-native shape, including `analysis`; the CLI does not rename those facts. +Raw JSON preserves the provider response. The deprecated `--agent` option +selects the legacy compact API shape. The focal comparison stays in its API-native +shape, including `analysis`; the CLI does not rename those facts. Similar fingerprint history and recent same-name history stay in separate paged collections. -Agent comparison JSON: +Legacy compact comparison JSON: ```json { @@ -1038,8 +1049,8 @@ vizzly status --json }, "suggestedCommands": [ { - "label": "Inspect build context", - "command": "vizzly --json context build abc123-def456 --agent --source cloud" + "label": "Discover build context API", + "command": "vizzly api schema sdk.getBuildContext --json" }, { "label": "List comparisons", diff --git a/skills/vizzly/SKILL.md b/skills/vizzly/SKILL.md index 04995879..a8651db5 100644 --- a/skills/vizzly/SKILL.md +++ b/skills/vizzly/SKILL.md @@ -24,23 +24,25 @@ For cloud API queries, selectable fields, or an authorized review decision, start with `vizzly api schema --json` and follow [schema discovery](references/cli-context.md#query-the-cloud-api). Discover request details as needed instead of loading the entire OpenAPI document. -The `context` commands below remain useful for local evidence and guided inspection. +Use `vizzly context ... --json` for local evidence and human-readable `context` +output for guided inspection. 1. Choose the supplied cloud build or comparison when one is named. Otherwise, use current local evidence or find the relevant cloud build. -2. Request bounded JSON: +2. For cloud data, discover the relevant API operations. For locally saved data, + request JSON context: ```bash - vizzly context build current --source local --agent --json - vizzly context build --source cloud --agent --json + vizzly api schema --json + vizzly context build current --source local --json ``` 3. Confirm the build, source, branch, timestamps, baseline, review state, and - pagination before drawing conclusions. If `has_more` is true, run the - returned next-page command before concluding. Missing fields remain unknown. -4. Follow `suggested_commands` to inspect a comparison. View its baseline, - current, and diff images together. If an image cannot be opened, label the - result metadata-only; do not call it visual verification. + pagination before drawing conclusions. Follow the schema's page instructions + for cloud data. Missing fields remain unknown. +4. For cloud data, discover the comparison and image operations from the schema. + View the baseline, current, and diff images together. If an image cannot be + opened, label the result metadata-only; do not call it visual verification. 5. Read image dimensions, viewport, browser, diff regions, fingerprint, and relevant history alongside the images. A prior approval is supporting evidence, not permission to approve the current comparison. diff --git a/skills/vizzly/references/cli-context.md b/skills/vizzly/references/cli-context.md index 27154b7e..fe15525d 100644 --- a/skills/vizzly/references/cli-context.md +++ b/skills/vizzly/references/cli-context.md @@ -47,17 +47,18 @@ builds and select the one matching the branch, commit, or pull request: ```bash vizzly builds --branch --limit 5 --json vizzly status --json -vizzly context build --source cloud --agent --json +vizzly api schema sdk.getBuildContext --json ``` -Use status for lifecycle facts and build context for visual evidence. Do not -assume the first returned comparison is the most important; preserve API order -and inspect the records relevant to the task. +Use status for lifecycle facts, then follow the discovered build-context +operation for visual evidence. Do not assume the first returned comparison is +the most important; preserve API order and inspect the records relevant to the +task. For saved local evidence: ```bash -vizzly context build current --source local --agent --json +vizzly context build current --source local --json vizzly context screenshot "" --source local --json vizzly context review-queue --source local --json ``` @@ -89,16 +90,16 @@ When a cloud build is in scope: ```bash vizzly run "" --wait --json -vizzly context build --source cloud --agent --json +vizzly api schema --json ``` ## Inspect A Comparison -Follow the build response's `suggested_commands`. The direct form is: +Discover comparison and image operations from the schema: ```bash -vizzly context comparison --source --agent --json -vizzly context comparison --source --agent --include diffs --json +vizzly api schema getComparisonContext --json +vizzly api schema sdk.getComparisonImage --json ``` Open all three images together. Prefer `original_url` and fall back to `url`: @@ -121,12 +122,10 @@ vizzly context similar --source cloud --json vizzly context review-queue --source --json ``` -Use `--include diffs` only when compact diagnostics are insufficient. Request -comments only when human review context matters. +Use the documented response and pagination fields to request only the evidence +needed. Request comments only when human review context matters. ## Continue Without Guessing -Run returned `suggested_commands` rather than reconstructing IDs, sources, or -pagination. When more evidence exists, the next-page command carries the API's -opaque `--cursor`; do not edit or interpret it. Keep follow-up commands pinned -to the source that produced the evidence. +Use IDs and pagination values from API responses. Keep cursors opaque and +preserve the query they belong to when requesting the next page. diff --git a/skills/vizzly/references/setup-ci.md b/skills/vizzly/references/setup-ci.md index 84eee09a..eafd3ebe 100644 --- a/skills/vizzly/references/setup-ci.md +++ b/skills/vizzly/references/setup-ci.md @@ -36,8 +36,8 @@ vizzly finalize "" --json - `vizzly doctor`: local configuration - `vizzly tdd status --json`: a local daemon started by the task - `vizzly status --json`: cloud lifecycle -- `vizzly context build --source cloud --agent --json`: visual - evidence +- `vizzly api schema --json`: discover supported cloud review operations and + their request and response schemas. If screenshots are absent, verify the existing integration, the test path that should capture them, and the active session. If authentication is absent, diff --git a/src/cli.js b/src/cli.js index bfcc8b2d..965c80ec 100644 --- a/src/cli.js +++ b/src/cli.js @@ -475,6 +475,16 @@ function getGlobalOptions() { }; } +function warnDeprecatedContextAgentOption() { + let warning = [ + '`vizzly context --agent` is deprecated and will be removed in v0.38.0.', + 'For cloud reviews, use `vizzly api schema --json` and follow the', + 'discovered operations. For local evidence, use', + '`vizzly context ... --json`.', + ].join(' '); + output.warn(warning); +} + function reportValidationErrors(errors) { if (output.isJson()) { output.error('Validation errors', null, { errors }); @@ -984,7 +994,7 @@ contextCmd .description('Fetch build context') .argument('', 'Build ID to fetch context for') .option('--source ', 'Context source: auto, cloud, or local', 'auto') - .option('--agent', 'Output compact context for LLM agents') + .option('--agent', 'Deprecated; use API schema for cloud review.') .option('--full', 'Return the full build context instead of compact context') .option( '--cursor ', @@ -1000,14 +1010,17 @@ contextCmd Examples: $ vizzly context build abc123 --source cloud $ vizzly context build current --source local - $ vizzly context build current --source local --agent - $ vizzly context build abc123 --source cloud --agent --json - $ vizzly context build abc123 --source cloud --agent --json --include diffs - $ vizzly context build abc123 --source cloud --agent --json --full + $ vizzly api schema --json + $ vizzly context build current --source local --json + +Deprecation: + --agent will be removed in v0.38.0. Use the API schema for cloud reviews; + use --json without --agent for local evidence. ` ) .action(async (buildId, options) => { let globalOptions = getGlobalOptions(); + if (options.agent) warnDeprecatedContextAgentOption(); const validationErrors = validateContextBuildOptions(options); if (validationErrors.length > 0) { reportValidationErrors(validationErrors); @@ -1021,7 +1034,7 @@ contextCmd .description('Fetch a comparison context bundle') .argument('', 'Comparison ID to fetch context for') .option('--source ', 'Context source: auto, cloud, or local', 'auto') - .option('--agent', 'Output compact context for LLM agents') + .option('--agent', 'Deprecated; use API schema for cloud review.') .option( '--full', 'Return the full comparison context instead of compact context' @@ -1038,11 +1051,16 @@ Examples: $ vizzly context comparison def456 --source cloud $ vizzly context comparison def456 --source local $ vizzly context comparison def456 --source cloud --json - $ vizzly context comparison def456 --source cloud --agent --json + $ vizzly api schema getComparisonContext --json + +Deprecation: + --agent will be removed in v0.38.0. Use the API schema for cloud reviews; + use --json without --agent for local evidence. ` ) .action(async (comparisonId, options) => { let globalOptions = getGlobalOptions(); + if (options.agent) warnDeprecatedContextAgentOption(); const validationErrors = validateContextComparisonOptions(options); if (validationErrors.length > 0) { reportValidationErrors(validationErrors); diff --git a/src/commands/init.js b/src/commands/init.js index c983c8dd..20f2565b 100644 --- a/src/commands/init.js +++ b/src/commands/init.js @@ -293,8 +293,10 @@ For user-facing changes, use the repo-local Vizzly skill at - Inspect existing visual evidence before and after the change. - Use the repository's established Vizzly command and owning user workflow. -- Read bounded JSON with \`--agent --json\`, then inspect baseline, current, and - diff images together. +- For cloud reviews, discover supported requests with + \`vizzly api schema --json\` and inspect baseline, current, and diff images + together. +- For local evidence, use \`vizzly context ... --json\`. Treat Vizzly diffs as review evidence. Do not approve, reject, or replace evidence unless the task explicitly asks for that mutation. diff --git a/src/commands/run.js b/src/commands/run.js index 7bf60584..3139ddf6 100644 --- a/src/commands/run.js +++ b/src/commands/run.js @@ -77,17 +77,17 @@ export async function resolveBuildDisplayUrl({ /** * Build the follow-up command for a cloud run. * - * JSON consumers need a self-contained command that returns structured - * evidence when executed. Human output keeps the existing readable summary. + * JSON consumers start with the public schema so they can discover the current + * request shape. Human output keeps the readable context summary. * * @param {string} buildId - Cloud build ID to inspect. * @param {Object} options - Command output options. * @param {boolean} [options.structured=false] - Include machine-readable JSON. - * @returns {string} Executable build context command. + * @returns {string} Cloud context or schema discovery command. */ function buildContextCommand(buildId, { structured = false } = {}) { - let jsonFlag = structured ? ' --json' : ''; - return `vizzly context build ${buildId} --agent${jsonFlag} --source cloud`; + if (structured) return 'vizzly api schema sdk.getBuildContext --json'; + return `vizzly context build ${buildId} --source cloud`; } /** diff --git a/src/commands/status.js b/src/commands/status.js index b5145328..e4d0dc67 100644 --- a/src/commands/status.js +++ b/src/commands/status.js @@ -103,8 +103,8 @@ export function createStatusSuggestedCommands(build = {}) { return [ { - label: 'Inspect build context', - command: `vizzly --json context build ${build.id} --agent --source cloud`, + label: 'Discover build context API', + command: 'vizzly api schema sdk.getBuildContext --json', }, { label: 'List comparisons', diff --git a/src/commands/tdd.js b/src/commands/tdd.js index e61c58f0..161a0c65 100644 --- a/src/commands/tdd.js +++ b/src/commands/tdd.js @@ -43,7 +43,7 @@ import * as defaultOutput from '../utils/output.js'; */ function buildLocalContextCommand({ structured = false } = {}) { let jsonFlag = structured ? ' --json' : ''; - return `vizzly context build current --source local --agent${jsonFlag}`; + return `vizzly context build current --source local${jsonFlag}`; } /** diff --git a/tests/cli/tdd-lifecycle.test.js b/tests/cli/tdd-lifecycle.test.js index 53f9a663..3405efef 100644 --- a/tests/cli/tdd-lifecycle.test.js +++ b/tests/cli/tdd-lifecycle.test.js @@ -143,7 +143,7 @@ describe('cli/tdd lifecycle', () => { assert.strictEqual(payload.data.summary.total, 0); assert.strictEqual( payload.data.contextCommand, - 'vizzly context build current --source local --agent --json' + 'vizzly context build current --source local --json' ); }); diff --git a/tests/commands/context-cli.test.js b/tests/commands/context-cli.test.js index 37cfb88c..2648887e 100644 --- a/tests/commands/context-cli.test.js +++ b/tests/commands/context-cli.test.js @@ -486,6 +486,46 @@ async function withBuildContextApi(callback) { } describe('context CLI integration', () => { + it('marks compact --agent context as deprecated without changing JSON stdout', async () => { + await withBuildContextApi(async ({ apiUrl }) => { + let result = await runCLI( + ['--json', 'context', 'build', 'build-123', '--agent'], + { + cwd: mkdtempSync(join(tmpdir(), 'vizzly-context-deprecation-')), + env: { + VIZZLY_API_URL: apiUrl, + VIZZLY_TOKEN: 'vzt_test_token', + }, + } + ); + + assert.strictEqual(result.code, 0, result.stderr); + let messages = parseJSONOutput(result.stderr); + assert.ok( + messages.some( + message => + message.status === 'warning' && + message.message.includes('will be removed in v0.38.0') && + message.message.includes('vizzly api schema --json') + ) + ); + assert.strictEqual( + JSON.parse(result.stdout).data.resource, + 'build_agent_context' + ); + }); + }); + + it('shows the removal plan in both context command help pages', async () => { + let buildHelp = await runCLI(['context', 'build', '--help']); + let comparisonHelp = await runCLI(['context', 'comparison', '--help']); + + assert.match(buildHelp.stdout, /Deprecated/); + assert.match(buildHelp.stdout, /v0\.38\.0/); + assert.match(comparisonHelp.stdout, /Deprecated/); + assert.match(comparisonHelp.stdout, /v0\.38\.0/); + }); + it('reports the resolved API origin when cloud context is unreachable', async () => { let server = createServer(request => request.socket.destroy()); await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); @@ -729,17 +769,23 @@ describe('context CLI integration', () => { assert.strictEqual(oversized.code, 1); assert.strictEqual( - JSON.parse(oversized.stderr).error.code, + parseJSONOutput(oversized.stderr).find( + message => message.status === 'error' + ).error.code, 'COMPACT_CONTEXT_OVERSIZED' ); assert.strictEqual(invalid.code, 1); assert.strictEqual( - JSON.parse(invalid.stderr).error.code, + parseJSONOutput(invalid.stderr).find( + message => message.status === 'error' + ).error.code, 'COMPACT_CONTEXT_INVALID' ); assert.strictEqual(nearLimit.code, 1); assert.strictEqual( - JSON.parse(nearLimit.stderr).error.code, + parseJSONOutput(nearLimit.stderr).find( + message => message.status === 'error' + ).error.code, 'COMPACT_CONTEXT_OVERSIZED' ); }); diff --git a/tests/commands/init.test.js b/tests/commands/init.test.js index a9894dd9..5989ea5f 100644 --- a/tests/commands/init.test.js +++ b/tests/commands/init.test.js @@ -383,7 +383,8 @@ describe('commands/init', () => { assert.match(installedSkill, /name: vizzly/); assert.match(agentsContent, /Visual Testing With Vizzly/); assert.match(agentsContent, /.agents\/skills\/vizzly\/SKILL.md/); - assert.match(agentsContent, /--agent --json/); + assert.match(agentsContent, /vizzly api schema --json/); + assert.match(agentsContent, /vizzly context \.\.\. --json/); assert.ok( output.calls.some( call => diff --git a/tests/commands/run-cli.test.js b/tests/commands/run-cli.test.js index 6347b990..85572c10 100644 --- a/tests/commands/run-cli.test.js +++ b/tests/commands/run-cli.test.js @@ -179,7 +179,6 @@ describe('commands/run CLI', () => { assert.match(result.stderr, /child stderr noise/); assert.doesNotMatch(result.stdout, /Screenshots/); assert.doesNotMatch(result.stdout, /Results/); - assert.doesNotMatch(result.stdout, /Context/); let payload = parseSingleJson(result.stdout); assert.strictEqual(payload.status, 'data'); @@ -188,7 +187,7 @@ describe('commands/run CLI', () => { assert.strictEqual(payload.data.comparisons.total, 0); assert.strictEqual( payload.data.contextCommand, - 'vizzly context build build-123 --agent --json --source cloud' + 'vizzly api schema sdk.getBuildContext --json' ); assert.deepStrictEqual( requests.map(request => `${request.method} ${request.url}`), @@ -240,7 +239,7 @@ describe('commands/run CLI', () => { assert.match(payload.data.error.message, /503/); assert.strictEqual( payload.data.contextCommand, - 'vizzly context build build-123 --agent --json --source cloud' + 'vizzly api schema sdk.getBuildContext --json' ); }, { waitStatusCode: 503 } diff --git a/tests/commands/status.test.js b/tests/commands/status.test.js index b3db588f..d05f8dd5 100644 --- a/tests/commands/status.test.js +++ b/tests/commands/status.test.js @@ -225,8 +225,8 @@ describe('status display decisions', () => { it('creates executable visual-context follow-up commands', () => { assert.deepStrictEqual(createStatusSuggestedCommands(createBuild()), [ { - label: 'Inspect build context', - command: 'vizzly --json context build build-123 --agent --source cloud', + label: 'Discover build context API', + command: 'vizzly api schema sdk.getBuildContext --json', }, { label: 'List comparisons',