diff --git a/README.md b/README.md index a2e08b6b..d482e7a8 100644 --- a/README.md +++ b/README.md @@ -93,6 +93,11 @@ your key, never the key itself. See [project link and credentials](https://github.com/vercel-labs/deepsec/blob/main/docs/vercel-setup.md) for the full reference. +When running locally, `deepsec` can also reuse existing Claude, Codex, Pi, or +OpenCode provider authentication — including a local `opencode` CLI login +(`opencode` → `/connect`) — for evaluation-scale scans. + + If a `process` or `revalidate` run halts because the upstream credential ran out of quota or credits, deepsec stops gracefully and tells you where to top up. Re-run the same command afterward and it picks up diff --git a/docs/architecture.md b/docs/architecture.md index dd5af953..8a0fefd0 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -105,12 +105,13 @@ Generated specs must use unique slugs; collisions are rejected before scan. - **Outputs:** FileRecord `findings[]` populated, `status: "analyzed"`, `analysisHistory[]` appended. -Three agent backends are supported: +Four agent backends are supported: | `--agent` | SDK | Default model | |---|---|---| | `codex` (default) | `@openai/codex-sdk` | `gpt-5.5` | | `claude` | `@anthropic-ai/claude-agent-sdk` | `claude-opus-4-8` | +| `opencode` | `@opencode-ai/sdk/v2` | `anthropic/claude-opus-4-8` | | `pi` | `@earendil-works/pi-coding-agent` | `zai/glm-5.2` | Same prompt, same JSON output schema. You can mix backends within a diff --git a/docs/configuration.md b/docs/configuration.md index a525fe4f..65b40cac 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -31,7 +31,7 @@ see [`samples/webapp/deepsec.config.ts`](https://github.com/vercel-labs/deepsec/ | `projects` | `ProjectDeclaration[]` | The codebases deepsec knows about. | | `plugins` | `DeepsecPlugin[]` | Loaded in order; later plugins override single-slot capabilities. | | `matchers` | `{ only?: string[]; exclude?: string[] }` | Filter the matcher set used by `scan`. | -| `defaultAgent` | `string` | Default `--agent` value (`codex`, `claude`, or `pi`). See [models](models.md). | +| `defaultAgent` | `string` | Default `--agent` value (`codex`, `claude`, `opencode`, or `pi`). See [models](models.md). | | `defaultModel` | `string` | Default `--model` value selected during setup. | | `defaultThinkingLevel` | `string` | Default reasoning effort (`minimal` through `xhigh`) selected during setup. | | `ai` | `ModelRoute` | Non-secret model credential route selected and verified by setup. | @@ -164,8 +164,9 @@ requires the complete access-token triple. | Var | Used by | Purpose | |---|---|---| -| `AI_GATEWAY_API_KEY` | Gateway route | Optional long-lived alternative to linked-project OIDC. Expanded for the selected agent. | -| `ANTHROPIC_AUTH_TOKEN` | `process`, `revalidate`, `triage` (Claude backend) | API token for the Claude Agent SDK. AI Gateway-issued or Anthropic-issued. Set this if you don't use `AI_GATEWAY_API_KEY`. | +| `AI_GATEWAY_API_KEY` | Gateway route | Optional long-lived alternative to linked-project OIDC. Expanded for the selected agent (Claude, Codex, OpenCode, or Pi). | +| `ANTHROPIC_AUTH_TOKEN` | `process`, `revalidate`, `triage` (Claude or OpenCode Anthropic backend) | API token for the Claude Agent SDK or the OpenCode Anthropic provider. AI Gateway-issued or Anthropic-issued. Set this if you don't use `AI_GATEWAY_API_KEY`. | +| `ANTHROPIC_API_KEY` | same | Direct-Anthropic credential (`x-api-key` broker contract). Prefer this over `ANTHROPIC_AUTH_TOKEN` when pointing directly at `https://api.anthropic.com` — some tooling rejects bearer-style tokens from that endpoint. | | `ANTHROPIC_BASE_URL` | same | Default (when `AI_GATEWAY_API_KEY` is set): `https://ai-gateway.vercel.sh`. Set to `https://api.anthropic.com` for direct Anthropic. | | `` | Direct/custom route | User-chosen variable containing the provider credential. The name is stored in config; the value comes from `.env.local` or the process. | @@ -173,10 +174,11 @@ requires the complete access-token triple. | Var | Used by | Purpose | |---|---|---| -| `OPENAI_API_KEY` | `--agent codex`, `--agent pi --model openai/...` | Codex SDK token or Pi OpenAI-provider token. Unset is fine if `AI_GATEWAY_API_KEY` is set. | -| `OPENAI_BASE_URL` | `--agent codex` | Default (when `AI_GATEWAY_API_KEY` is set): `https://ai-gateway.vercel.sh/v1`. | +| `OPENAI_API_KEY` | `--agent codex`, `--agent opencode --model openai/...`, `--agent pi --model openai/...` | Codex SDK, OpenCode, or Pi OpenAI-provider token. Unset is fine if `AI_GATEWAY_API_KEY` is set. | +| `OPENAI_BASE_URL` | `--agent codex`, `--agent opencode --model openai/...` | Default (when `AI_GATEWAY_API_KEY` is set): `https://ai-gateway.vercel.sh/v1`. | | `PI_CODING_AGENT_DIR` | `--agent pi` | Optional Pi config/auth directory. Defaults to `~/.pi/agent`; local non-sandbox runs can reuse `auth.json` there. | -| `DEEPSEC_AGENT_DEBUG` | both backends | Set to `1` to enable verbose agent logging. | +| `XDG_DATA_HOME` | `--agent opencode` | Optional OpenCode data root. Local preflight looks for provider auth at `$XDG_DATA_HOME/opencode/auth.json` (default `~/.local/share/opencode/auth.json`). | +| `DEEPSEC_AGENT_DEBUG` | AI backends | Set to `1` to enable verbose agent logging. | | `DEEPSEC_DATA_ROOT` | core | Override the data directory location. Equivalent to `dataDir` in config. | ### Plugin-specific diff --git a/docs/data-layout.md b/docs/data-layout.md index db63d6b8..6f908ff6 100644 --- a/docs/data-layout.md +++ b/docs/data-layout.md @@ -165,7 +165,7 @@ deleted. | `investigatedAt` | `string` (ISO) | Timestamp. | | `durationMs` | `number` | Wall-clock total. | | `durationApiMs` | `number?` | API time only (excludes process orchestration). | -| `agentType` | `string` | `claude-agent-sdk` or `codex`. | +| `agentType` | `string` | Producing backend: `claude-agent-sdk`, `codex`, `opencode`, `pi`, or a plugin agent type. | | `model` | `string` | Model identifier. | | `modelConfig` | `Record` | Provider-specific settings echoed back. | | `agentSessionId` | `string?` | The agent's session/thread id, for reproducing or replaying. | diff --git a/docs/models.md b/docs/models.md index 791c33ae..dad15580 100644 --- a/docs/models.md +++ b/docs/models.md @@ -1,16 +1,17 @@ --- title: "Models" -description: "Choose Codex, Claude, or Pi for process and revalidate runs, and compare models under the same workload." +description: "Choose Codex, Claude, OpenCode, or Pi for process and revalidate runs, and compare models under the same workload." --- deepsec talks to LLMs through interchangeable agent backends: -| Backend | Default model | Used by | -|-----------------------------|-----------------------|------------------------------| -| `codex` (default) | `gpt-5.5` | `process`, `revalidate` | -| `claude` | `claude-opus-4-8` | `process`, `revalidate` | -| `pi` | `zai/glm-5.2` | `process`, `revalidate` | -| `claude` (triage) | `claude-sonnet-4-6` | `triage` (Claude-only) | +| Backend | Default model | Used by | +|-----------------------------|---------------------------------|-------------------------| +| `codex` (default) | `gpt-5.5` | `process`, `revalidate` | +| `claude` | `claude-opus-4-8` | `process`, `revalidate` | +| `opencode` | `anthropic/claude-opus-4-8` | `process`, `revalidate` | +| `pi` | `zai/glm-5.2` | `process`, `revalidate` | +| `claude` (triage) | `claude-sonnet-4-6` | `triage` (Claude-only) | Interactive one-shot setup recommends five benchmark-backed combinations: GPT-5.6 Sol, Claude Opus 5, Kimi K3, Grok 4.5, and the current DeepSeek entry. @@ -55,6 +56,12 @@ disables the env-var preflight checks — the machine-wide login is used directly. Sandbox commands are the exception: they must broker a real token, so they still require `AI_GATEWAY_API_KEY` (or an equivalent key). + +The built-in backends also work with direct `AI_GATEWAY_API_KEY` credentials: +one key covers Codex, Claude, OpenCode, and Pi. OpenCode and Pi accept +provider/model identifiers directly, which makes them useful for comparing +harness and provider behavior under the same deepsec workload. + ## CLI selection ```bash @@ -70,6 +77,10 @@ pnpm deepsec process --project-id my-app --agent codex # Codex backend, specific model: pnpm deepsec process --project-id my-app --agent codex --model gpt-5.4 +# OpenCode SDK harness (provider/model is required): +pnpm deepsec process --project-id my-app --agent opencode +pnpm deepsec process --project-id my-app --agent opencode --model openai/gpt-5.5 + # Pi backend through Vercel AI Gateway, default model: pnpm deepsec process --project-id my-app --agent pi @@ -101,11 +112,12 @@ over large repos. The flag maps onto each backend's native dial: -| Backend | Setting | -|----------|---------------------------------------------| -| `codex` | model reasoning effort (`minimal`–`xhigh`) | -| `pi` | thinking level (`minimal`–`xhigh`) | -| `claude` | adaptive-thinking effort (`minimal` → `low`, `xhigh` → `max`) | +| Backend | Setting | +|------------|---------------------------------------------------------------| +| `codex` | model reasoning effort (`minimal`–`xhigh`) | +| `opencode` | provider variant (Anthropic maps to `high` / `max`) | +| `pi` | thinking level (`minimal`–`xhigh`) | +| `claude` | adaptive-thinking effort (`minimal` → `low`, `xhigh` → `max`) | It applies to the main investigation/revalidation runs only. Special-purpose follow-up calls (the refusal report, JSON repair) keep @@ -171,6 +183,62 @@ Later `process`, `revalidate`, and Sandbox commands resolve the persisted route. Per-command `--ai-provider`, `--ai-base-url`, `--ai-api-key-env`, and repeatable `--ai-header name=value` remain available as Pi runtime overrides. +### OpenCode SDK harness + +OpenCode uses `@opencode-ai/sdk/v2` plus the `opencode` runtime — both the +v1 runtime (npm `opencode-ai` 1.x, bundled for sandbox workers) and the +**OpenCode v2** runtime (2.0.x) are supported; the generation is detected at +startup and the matching protocol is used automatically. Each batch starts a +private, password-secured local OpenCode server, creates a session rooted at +the target project, and asks for JSON output. The deepsec agent allows only +`read`, `glob`, `grep`, and `list`; shell, edits, network tools, subagents, +external directories, LSP, skills, and MCP-triggered permissions are denied. +The session and server are closed on success, error, or abort. + +Differences worth knowing: + +- The v1 runtime returns JSON Schema–validated structured output when the + model supports it; the v2 runtime has no structured-output prompts, so + results arrive as plain text and go through deepsec's shared JSON-repair + pipeline (same contract, same validation). +- The v2 runtime always requires basic auth on its server; deepsec generates + the credential itself and nothing is written to disk. +- `--ai-header` overrides require the v1 runtime; on v2 they are rejected + with a clear error. + +**Vercel AI Gateway on the v2 runtime** — two supported options: + +- Native gateway provider (recommended): the v2 model catalog ships a + `vercel` provider ("Vercel AI Gateway") that reads `AI_GATEWAY_API_KEY` + directly — one key covers every model behind the gateway: + + ```bash + AI_GATEWAY_API_KEY=vck_… pnpm deepsec process --project-id my-app \ + --agent opencode \ + --model vercel/anthropic/claude-opus-4-8 + ``` + +- The standard `ANTHROPIC_*`/`OPENAI_*` expansion also keeps working: + deepsec translates the expansion into the v2 provider `baseURL` overlays + and bridges the gateway bearer token to the `x-api-key` form the v2 + anthropic provider reads, so `--model anthropic/claude-opus-4-8` with + `AI_GATEWAY_API_KEY` routes through the gateway unchanged. + +Models use OpenCode's required `provider/model` form: + +```bash +pnpm deepsec process --project-id my-app \ + --agent opencode \ + --model anthropic/claude-opus-4-8 +``` + +For a local run without environment credentials, authenticate a provider in +OpenCode first: run `opencode`, then use `/connect`. Sandbox runs cannot reuse +that local credential store; use AI Gateway or an explicit provider token. +Generic `--ai-provider`, `--ai-base-url`, `--ai-api-key-env`, and +`--ai-header` overrides work for OpenCode as they do for Pi (headers v1 +runtime only). + ### `claude-sonnet-4-6` for `triage` Triage buckets findings into P0/P1/P2/skip without re-reading the code. @@ -216,6 +284,7 @@ rest of deepsec stays unchanged: ```bash pnpm deepsec process --project-id my-app --model anthropic-mythos-1 pnpm deepsec process --project-id my-app --agent codex --model gpt-6 +pnpm deepsec process --project-id my-app --agent opencode --model anthropic/claude-mythos-1 pnpm deepsec process --project-id my-app --agent pi --model vercel-ai-gateway/openai/gpt-6 ``` diff --git a/docs/vercel-setup.md b/docs/vercel-setup.md index d65a4865..4ffc4559 100644 --- a/docs/vercel-setup.md +++ b/docs/vercel-setup.md @@ -100,7 +100,7 @@ ai: { mode: "gateway", provider: "vercel" } ``` Deepsec maps the Gateway credential to the environment expected by Codex, -Claude, or Pi. Sandbox workers receive only a placeholder. The real bearer +Claude, OpenCode, or Pi. One key covers all four harnesses. Sandbox workers receive only a placeholder. The real bearer token is injected at the allowed Gateway host by the host-side broker. ### Your own OpenAI or Anthropic credential @@ -133,13 +133,14 @@ again or put it in `.deepsec/.env.local`: MY_OPENAI_KEY=... ``` -Direct OpenAI routes require Codex; direct Anthropic routes require Claude. +Direct OpenAI routes require Codex or OpenCode (`openai/...` models); direct +Anthropic routes require Claude or OpenCode (`anthropic/...` models). Setup rejects incompatible agent/provider combinations before scanning. ### Custom HTTPS provider -Custom routes are supported by the Pi backend. Supply an HTTPS base URL and -describe how the credential should be attached: +Custom routes are supported by the Pi and OpenCode backends. Supply an HTTPS +base URL and describe how the credential should be attached: ```bash MARTIAN_KEY=... npx deepsec init \ @@ -156,6 +157,29 @@ Use `:bearer` for `Authorization: Bearer …` and `:raw` for provider-specific raw token headers. HTTP URLs, embedded URL credentials, invalid header names, and custom routes for Codex/Claude are rejected. +OpenCode uses `provider/model` identifiers, so the same custom route works +there too: + +```bash +MARTIAN_KEY=... npx deepsec init \ + --agent opencode \ + --model openai/gpt-5.5 \ + --model-auth custom \ + --ai-provider martian \ + --ai-api-key-env MARTIAN_KEY \ + --ai-base-url https://api.martian.example/v1 \ + --ai-credential-header authorization:bearer +``` + +### Local agent authentication + +Non-sandbox runs can reuse an existing local login for the built-in +harnesses: `claude login` / `codex login` for those CLIs, or run `opencode` +and use `/connect` to connect a provider for the OpenCode backend. Local +OpenCode auth (stored under `$XDG_DATA_HOME/opencode/auth.json`) is never +copied into Vercel Sandbox workers — sandbox runs need the Gateway (or a +direct/custom route). + ## Credential brokering in Sandbox Deepsec resolves the real model credential on the host. A worker receives a @@ -211,5 +235,6 @@ pnpm deepsec sandbox-all process --sandboxes 30 --concurrency 4 | Direct route asks for a custom variable | Config stores its name, not its value. | Export it in the fresh shell or add it to `.env.local`. | | Setup repeats login verification | The link, route, agent, or verification age changed. | Complete the probe; later unchanged runs short-circuit again. | | Gateway quota is exhausted | Processing stopped before launching more batches. | Add credits or change route, then re-run; processing resumes pending files. | +| `Missing AI credentials for --agent opencode` | No Gateway/direct credential and no local OpenCode auth. | Set `AI_GATEWAY_API_KEY`, use a direct route, or run `opencode` and use `/connect` (local runs only). | References: [AI Gateway authentication](https://vercel.com/docs/ai-gateway/authentication-and-byok#quick-start), [Sandbox authentication](https://vercel.com/docs/vercel-sandbox/concepts/authentication). diff --git a/e2e/bundle.test.ts b/e2e/bundle.test.ts index a3920f2d..2057cef2 100644 --- a/e2e/bundle.test.ts +++ b/e2e/bundle.test.ts @@ -47,6 +47,10 @@ describe("bundle e2e", () => { expect(stdout).toContain("deepsec"); expect(stdout).toContain("scan"); expect(stdout).toContain("process"); + + const processHelp = runBundle(["process", "--help"]); + expect(processHelp.status).toBe(0); + expect(processHelp.stdout).toContain("opencode"); }); it("config.d.ts is self-contained (no internal @deepsec/* re-exports)", () => { @@ -186,7 +190,9 @@ export default defineConfig({ expect(stdout).toContain("webapp-debug-flag"); expect(stdout).toContain("webapp-route-no-rate-limit"); } finally { - fs.rmSync(link, { force: true }); + try { + fs.unlinkSync(link); + } catch {} fs.rmSync(path.join(sampleDir, "data"), { recursive: true, force: true }); } }); diff --git a/e2e/pipeline-sandbox.test.ts b/e2e/pipeline-sandbox.test.ts index 42c661bb..c10680d5 100644 --- a/e2e/pipeline-sandbox.test.ts +++ b/e2e/pipeline-sandbox.test.ts @@ -64,6 +64,7 @@ const HAS_SANDBOX_KEY = Boolean(process.env.VERCEL_PROJECT_ID)); const SHOULD_RUN = LIVE && HAS_SANDBOX_KEY; +const SANDBOX_TIMEOUT_MS = 30 * 60_000; /** * Stub agent definition shared with pipeline.test.ts. Inlined here @@ -319,6 +320,8 @@ describe.skipIf(!SHOULD_RUN)("pipeline e2e — live sandbox", () => { "1", "--vcpus", "2", + "--timeout", + String(SANDBOX_TIMEOUT_MS), "--limit", "2", "--concurrency", @@ -358,6 +361,8 @@ describe.skipIf(!SHOULD_RUN)("pipeline e2e — live sandbox", () => { "1", "--vcpus", "2", + "--timeout", + String(SANDBOX_TIMEOUT_MS), "--limit", "2", "--concurrency", diff --git a/knip.json b/knip.json index baf2f445..e6e10415 100644 --- a/knip.json +++ b/knip.json @@ -35,6 +35,9 @@ "@anthropic-ai/claude-agent-sdk", "@earendil-works/pi-coding-agent", "@openai/codex", - "@openai/codex-sdk" + "@openai/codex-sdk", + "@opencode-ai/sdk", + "opencode-ai", + "undici" ] } diff --git a/packages/core/src/__tests__/fs-settle.test.ts b/packages/core/src/__tests__/fs-settle.test.ts new file mode 100644 index 00000000..7282415b --- /dev/null +++ b/packages/core/src/__tests__/fs-settle.test.ts @@ -0,0 +1,129 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { existsSettled, mkdirSettled, settleBudgetMs, settleFs } from "../fs-settle.js"; + +/** + * Docker Desktop / virtiofs bind mounts of a macOS host return from a create + * syscall before the new entry is visible to path lookup — measured at 1-2ms, + * and 100% reproducible inside that window. These tests fake that lag, since + * a real non-coherent mount is not available in CI. + */ +function enoent(target: string): NodeJS.ErrnoException { + const error: NodeJS.ErrnoException = new Error( + `ENOENT: no such file or directory, mkdir '${target}'`, + ); + error.code = "ENOENT"; + return error; +} + +const NO_WAIT = { DEEPSEC_FS_SETTLE_MS: "0" }; + +afterEach(() => { + vi.restoreAllMocks(); +}); + +describe("fs-settle", () => { + // The failure that broke `deepsec init` on a mounted host filesystem: + // recursive mkdir creates the parent, then cannot see it when it looks up + // the child, and reports `ENOENT: mkdir 'data/'` for a path + // whose parent it just created. + it("mkdirSettled retries a recursive mkdir that fails on its own fresh parent", () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "settle-")); + const target = path.join(root, "data", "my-project"); + let calls = 0; + const real = fs.mkdirSync; + vi.spyOn(fs, "mkdirSync").mockImplementation(((p: string, opts: never) => { + calls += 1; + if (calls === 1) throw enoent(p); + return real(p, opts); + }) as typeof fs.mkdirSync); + mkdirSettled(target, NO_WAIT); + expect(calls).toBe(2); + expect(fs.existsSync(target)).toBe(true); + }); + + it("mkdirSettled creates the directory when nothing lags", () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "settle-")); + const target = path.join(root, "data", "my-project"); + mkdirSettled(target, NO_WAIT); + expect(fs.existsSync(target)).toBe(true); + }); + + // A permission error is not a timing problem — retrying just doubles the + // wait before reporting the same failure. + it("mkdirSettled rethrows a non-ENOENT error without retrying", () => { + let calls = 0; + vi.spyOn(fs, "mkdirSync").mockImplementation((() => { + calls += 1; + const error: NodeJS.ErrnoException = new Error("EACCES: permission denied, mkdir '/nope'"); + error.code = "EACCES"; + throw error; + }) as typeof fs.mkdirSync); + expect(() => mkdirSettled("/nope/deeper", NO_WAIT)).toThrow(/EACCES/); + expect(calls).toBe(1); + }); + + it("mkdirSettled surfaces ENOENT when the retry also fails", () => { + vi.spyOn(fs, "mkdirSync").mockImplementation(((p: string) => { + throw enoent(p); + }) as unknown as typeof fs.mkdirSync); + expect(() => mkdirSettled("/nope/deeper", NO_WAIT)).toThrow(/ENOENT/); + }); + + it("existsSettled re-checks a lagging lookup once", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "settle-")); + let calls = 0; + const real = fs.existsSync; + vi.spyOn(fs, "existsSync").mockImplementation((target) => { + calls += 1; + return calls === 1 ? false : real(target); + }); + expect(existsSettled(dir, NO_WAIT)).toBe(true); + expect(calls).toBe(2); + }); + + it("existsSettled costs one call when the path is already visible", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "settle-")); + const spy = vi.spyOn(fs, "existsSync"); + expect(existsSettled(dir, NO_WAIT)).toBe(true); + expect(spy).toHaveBeenCalledTimes(1); + }); + + it("existsSettled reports a genuinely missing path as missing", () => { + expect(existsSettled(path.join(os.tmpdir(), "settle-does-not-exist-42"), NO_WAIT)).toBe(false); + }); + + describe("settleFs", () => { + it("waits the configured time", () => { + const started = Date.now(); + settleFs({ DEEPSEC_FS_SETTLE_MS: "40" }); + expect(Date.now() - started).toBeGreaterThanOrEqual(35); + }); + + it("returns immediately when the wait is disabled", () => { + const started = Date.now(); + settleFs(NO_WAIT); + expect(Date.now() - started).toBeLessThan(20); + }); + }); + + describe("settleBudgetMs", () => { + it("defaults to 100ms", () => { + expect(settleBudgetMs({})).toBe(100); + }); + + it("honors DEEPSEC_FS_SETTLE_MS for slower sync backends", () => { + expect(settleBudgetMs({ DEEPSEC_FS_SETTLE_MS: "500" })).toBe(500); + }); + + it("allows opting out entirely", () => { + expect(settleBudgetMs(NO_WAIT)).toBe(0); + }); + + it("rejects a nonsense value", () => { + expect(() => settleBudgetMs({ DEEPSEC_FS_SETTLE_MS: "soon" })).toThrow(/non-negative number/); + }); + }); +}); diff --git a/packages/core/src/__tests__/run.test.ts b/packages/core/src/__tests__/run.test.ts index 97b3f381..8589a4b1 100644 --- a/packages/core/src/__tests__/run.test.ts +++ b/packages/core/src/__tests__/run.test.ts @@ -202,6 +202,47 @@ describe("ensureProject", () => { } }); + // Regression: on a Docker/virtiofs bind mount of a macOS host, recursive + // mkdir creates `data/`, then cannot see it when it looks up the child, and + // reports `ENOENT: mkdir 'data/'`. ensureProject must survive it. + it("creates the project data dir on a filesystem with a lagging lookup", () => { + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-lagging-fs-")); + const oldDataRoot = process.env.DEEPSEC_DATA_ROOT; + process.env.DEEPSEC_DATA_ROOT = path.join(tmp, "data"); + const root = path.join(tmp, "project"); + fs.mkdirSync(root); + + let mkdirCalls = 0; + const realMkdir = fs.mkdirSync; + const mkdirSpy = vi.spyOn(fs, "mkdirSync").mockImplementation((( + target: string, + options: never, + ) => { + mkdirCalls += 1; + if (mkdirCalls === 1) { + const error: NodeJS.ErrnoException = new Error( + `ENOENT: no such file or directory, mkdir '${target}'`, + ); + error.code = "ENOENT"; + throw error; + } + return realMkdir(target, options); + }) as typeof fs.mkdirSync); + + try { + const project = ensureProject("test-project", root); + expect(project.projectId).toBe("test-project"); + expect(mkdirCalls).toBeGreaterThan(1); + mkdirSpy.mockRestore(); + expect(fs.existsSync(path.join(process.env.DEEPSEC_DATA_ROOT, "test-project"))).toBe(true); + } finally { + mkdirSpy.mockRestore(); + if (oldDataRoot === undefined) delete process.env.DEEPSEC_DATA_ROOT; + else process.env.DEEPSEC_DATA_ROOT = oldDataRoot; + fs.rmSync(tmp, { recursive: true, force: true }); + } + }); + it("does not print git errors for non-git roots", () => { const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-ensure-project-")); const oldDataRoot = process.env.DEEPSEC_DATA_ROOT; diff --git a/packages/core/src/__tests__/severity.test.ts b/packages/core/src/__tests__/severity.test.ts index 4faa6981..0d22ab1a 100644 --- a/packages/core/src/__tests__/severity.test.ts +++ b/packages/core/src/__tests__/severity.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from "vitest"; import { findingSchema, revalidationSchema } from "../schemas.js"; +import { SEVERITY_ORDER } from "../severity.js"; describe("severity levels", () => { const baseFinding = { @@ -38,6 +39,17 @@ describe("severity levels", () => { it("rejects empty severity", () => { expect(() => findingSchema.parse({ ...baseFinding, severity: "" })).toThrow(); }); + + it("uses one canonical ordering including LOW", () => { + expect(SEVERITY_ORDER).toEqual({ + CRITICAL: 0, + HIGH: 1, + MEDIUM: 2, + HIGH_BUG: 3, + BUG: 4, + LOW: 5, + }); + }); }); describe("revalidation adjustedSeverity", () => { diff --git a/packages/core/src/fs-settle.ts b/packages/core/src/fs-settle.ts new file mode 100644 index 00000000..6b498cc8 --- /dev/null +++ b/packages/core/src/fs-settle.ts @@ -0,0 +1,86 @@ +// Helpers for filesystems where a create syscall returns before the new +// entry is visible to path lookup. +// +// Observed on Docker Desktop / virtiofs bind mounts of a macOS host (the +// shape most agent sandboxes use): `mkdir`, `writeFile`, and `symlink` +// return success, but `lstat`/`realpath`/`existsSync` on the path they just +// created report ENOENT for the next 1-2ms. `open`/`read`/`write` against +// the same path work immediately — it is the lookup path that lags, not the +// data. Measured visibility: 0/10 at 0ms, 9/10 at 1ms, 10/10 at 2ms. +// +// POSIX says a create is durable when the syscall returns, so code that +// creates a directory and then stats it is correct and still breaks here. +// The fix is to wait out the lag. +// +// Use these ONLY right after this process created the path. Everywhere else +// the wait is pure cost: an absent path stays absent no matter how long you +// wait for it. + +import fs from "node:fs"; + +// 100ms is ~50x the observed virtiofs window. Deliberately a flat wait +// rather than a poll-until-visible loop: it is a handful of call sites on +// one code path (workspace scaffolding), and the simplicity is worth more +// than the milliseconds. +const DEFAULT_BUDGET_MS = 100; + +/** + * How long to wait for a create to become visible. Mutagen-based file + * sharing (Docker Desktop's "Synchronized file shares") can lag far longer + * than virtiofs, so the wait is tunable via `DEEPSEC_FS_SETTLE_MS`. Set it + * to 0 on a filesystem you know is coherent. + */ +export function settleBudgetMs(env: NodeJS.ProcessEnv = process.env): number { + const raw = env.DEEPSEC_FS_SETTLE_MS; + if (raw === undefined) return DEFAULT_BUDGET_MS; + const parsed = Number(raw); + if (!Number.isFinite(parsed) || parsed < 0) { + throw new Error(`DEEPSEC_FS_SETTLE_MS must be a non-negative number, got "${raw}"`); + } + return parsed; +} + +/** + * Wait out the mount's create-to-lookup lag. + * + * Every caller here is synchronous (`ensureProject`, workspace + * scaffolding), so a timer is not an option; `Atomics.wait` on an + * uncontended buffer parks the thread instead of spinning the CPU. + */ +export function settleFs(env?: NodeJS.ProcessEnv): void { + const ms = settleBudgetMs(env); + if (ms <= 0) return; + Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms); +} + +/** + * `fs.existsSync` for a path another process just created — a "false" is + * re-checked once after settling. + */ +export function existsSettled(target: string, env?: NodeJS.ProcessEnv): boolean { + if (fs.existsSync(target)) return true; + settleFs(env); + return fs.existsSync(target); +} + +/** + * Recursive `mkdir` that leaves the directory visible to path lookup when + * it returns. + * + * Node's recursive mkdir walks the path creating each missing component, + * and on a lagging mount the freshly created parent can still be invisible + * when it looks up the child — which surfaces as + * `ENOENT: mkdir '/'` for a path whose parent was just + * created successfully. So ENOENT here means "too early", not "missing": + * settle and try once more. + */ +export function mkdirSettled(target: string, env?: NodeJS.ProcessEnv): void { + try { + fs.mkdirSync(target, { recursive: true }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + settleFs(env); + fs.mkdirSync(target, { recursive: true }); + } + settleFs(env); +} diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index ea8dd86a..e0f7d039 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,7 +1,9 @@ export * from "./config.js"; export * from "./finding-id.js"; +export * from "./fs-settle.js"; export * from "./paths.js"; export * from "./plugin.js"; export * from "./run.js"; export * from "./schemas.js"; +export * from "./severity.js"; export * from "./types.js"; diff --git a/packages/core/src/run.ts b/packages/core/src/run.ts index a8ca222d..207bfbb4 100644 --- a/packages/core/src/run.ts +++ b/packages/core/src/run.ts @@ -4,6 +4,7 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; import { ensureFindingIds } from "./finding-id.js"; +import { mkdirSettled } from "./fs-settle.js"; import { dataDir, fileRecordPath, @@ -85,7 +86,7 @@ export function ensureProject(projectId: string, rootPath: string): ProjectConfi createdAt: new Date().toISOString(), githubUrl: detectGithubUrl(path.resolve(rootPath)), }; - fs.mkdirSync(path.dirname(configPath), { recursive: true }); + mkdirSettled(path.dirname(configPath)); writeProjectConfig(configPath, config); return config; } diff --git a/packages/core/src/severity.ts b/packages/core/src/severity.ts new file mode 100644 index 00000000..8a7d0caf --- /dev/null +++ b/packages/core/src/severity.ts @@ -0,0 +1,14 @@ +import type { Severity } from "./types.js"; + +/** + * Canonical severity ranking used for filtering and presentation. + * Lower numbers are more severe. + */ +export const SEVERITY_ORDER: Record = { + CRITICAL: 0, + HIGH: 1, + MEDIUM: 2, + HIGH_BUG: 3, + BUG: 4, + LOW: 5, +}; diff --git a/packages/deepsec/build.mjs b/packages/deepsec/build.mjs index 4360c99a..29b188a0 100644 --- a/packages/deepsec/build.mjs +++ b/packages/deepsec/build.mjs @@ -15,8 +15,10 @@ const external = [ "@earendil-works/pi-coding-agent", "@openai/codex", "@openai/codex-sdk", + "@opencode-ai/sdk", "@vercel/sandbox", "jiti", + "undici", ]; // Ink's devtools module imports this optional development-only package. The diff --git a/packages/deepsec/package.json b/packages/deepsec/package.json index 0b025f25..7a40bee0 100644 --- a/packages/deepsec/package.json +++ b/packages/deepsec/package.json @@ -37,13 +37,16 @@ "@earendil-works/pi-coding-agent": "0.81.1", "@openai/codex": "^0.156.1", "@openai/codex-sdk": "^0.156.1", + "@opencode-ai/sdk": "^1.18.35", "@vercel/oidc": "^3.4.0", "@vercel/sandbox": "^1.9.0", "ink": "^7.1.1", "jiti": "^2.4.0", "minimatch": "^10.0.0", + "opencode-ai": "^1.18.35", "react": "^19.2.8", - "tar": "^7.5.20" + "tar": "^7.5.20", + "undici": "^8.5.0" }, "devDependencies": { "@deepsec/core": "workspace:*", diff --git a/packages/deepsec/src/__tests__/agent-defaults.test.ts b/packages/deepsec/src/__tests__/agent-defaults.test.ts index fdb62658..f0f50c55 100644 --- a/packages/deepsec/src/__tests__/agent-defaults.test.ts +++ b/packages/deepsec/src/__tests__/agent-defaults.test.ts @@ -8,6 +8,7 @@ describe("defaultModelForAgent", () => { it("returns the backend-specific default models", () => { expect(defaultModelForAgent("codex")).toBe("gpt-5.5"); expect(defaultModelForAgent("pi")).toBe("zai/glm-5.2"); + expect(defaultModelForAgent("opencode")).toBe("anthropic/claude-opus-4-8"); expect(defaultModelForAgent("claude-agent-sdk")).toBe("claude-opus-4-8"); }); diff --git a/packages/deepsec/src/__tests__/credential-brokering.test.ts b/packages/deepsec/src/__tests__/credential-brokering.test.ts index 82b47b66..65b196e1 100644 --- a/packages/deepsec/src/__tests__/credential-brokering.test.ts +++ b/packages/deepsec/src/__tests__/credential-brokering.test.ts @@ -4,6 +4,7 @@ import { buildSandboxEnv, resolveBrokeredCredentials } from "../sandbox/setup.js const TOUCHED_KEYS = [ "AI_GATEWAY_API_KEY", "ANTHROPIC_AUTH_TOKEN", + "ANTHROPIC_API_KEY", "ANTHROPIC_BASE_URL", "OPENAI_API_KEY", "OPENAI_BASE_URL", @@ -36,6 +37,12 @@ describe("credential brokering", () => { expect(c.openaiToken).toBeUndefined(); }); + it("captures ANTHROPIC_API_KEY for the OpenCode sandbox path", () => { + process.env.ANTHROPIC_API_KEY = "sk-ant-real"; + const c = resolveBrokeredCredentials("opencode"); + expect(c.anthropicToken).toBe("sk-ant-real"); + }); + it("falls back to ANTHROPIC token for OpenAI on the codex path", () => { process.env.ANTHROPIC_AUTH_TOKEN = "vck_real"; const c = resolveBrokeredCredentials("codex"); @@ -66,6 +73,14 @@ describe("credential brokering", () => { const c = resolveBrokeredCredentials("pi", { aiApiKeyEnv: "MARTIAN_API_KEY" }); expect(c.customToken).toEqual({ envName: "MARTIAN_API_KEY", token: "martian-real" }); }); + + it("captures a custom API key env for the OpenCode path", () => { + process.env.MARTIAN_API_KEY = "martian-real"; + const c = resolveBrokeredCredentials("opencode", { + aiApiKeyEnv: "MARTIAN_API_KEY", + }); + expect(c.customToken).toEqual({ envName: "MARTIAN_API_KEY", token: "martian-real" }); + }); }); describe("buildSandboxEnv (the env handed to Sandbox.create)", () => { @@ -174,5 +189,38 @@ describe("credential brokering", () => { expect(v).not.toContain("martian-real-secret"); } }); + + it("routes OpenCode Anthropic traffic through the local proxy", () => { + process.env.ANTHROPIC_AUTH_TOKEN = "vck_real_opencode_secret"; + process.env.ANTHROPIC_BASE_URL = "https://ai-gateway.vercel.sh"; + const credentials = resolveBrokeredCredentials("opencode"); + const env = buildSandboxEnv("opencode", credentials, { + model: "anthropic/claude-opus-4-8", + }); + + expect(env.DEEPSEC_OPENCODE_PROVIDER).toBe("anthropic"); + expect(env.ANTHROPIC_AUTH_TOKEN).toBeDefined(); + expect(env.ANTHROPIC_AUTH_TOKEN).not.toBe("vck_real_opencode_secret"); + expect(env.ANTHROPIC_UPSTREAM_BASE_URL).toBe("https://ai-gateway.vercel.sh"); + expect(env.ANTHROPIC_BASE_URL).toMatch(/^http:\/\/127\.0\.0\.1:\d+$/); + }); + + it("routes a custom OpenCode provider without exposing its key", () => { + process.env.MARTIAN_API_KEY = "martian-real-secret"; + const credentials = resolveBrokeredCredentials("opencode", { + aiApiKeyEnv: "MARTIAN_API_KEY", + }); + const env = buildSandboxEnv("opencode", credentials, { + aiApiKeyEnv: "MARTIAN_API_KEY", + aiBaseUrl: "https://api.withmartian.com/v1", + aiProvider: "martian", + model: "openai/security-model", + }); + + expect(env.DEEPSEC_OPENCODE_PROVIDER).toBe("martian"); + expect(env.DEEPSEC_OPENCODE_AI_BASE_URL).toBe("https://api.withmartian.com/v1"); + expect(env.MARTIAN_API_KEY).toBeDefined(); + expect(env.MARTIAN_API_KEY).not.toBe("martian-real-secret"); + }); }); }); diff --git a/packages/deepsec/src/__tests__/ensure-connected-workspace.test.ts b/packages/deepsec/src/__tests__/ensure-connected-workspace.test.ts index 510ca515..bd3fc2c1 100644 --- a/packages/deepsec/src/__tests__/ensure-connected-workspace.test.ts +++ b/packages/deepsec/src/__tests__/ensure-connected-workspace.test.ts @@ -112,4 +112,91 @@ describe("ensureConnectedWorkspace", () => { VERCEL_PROJECT_ID: "project", }); }); + + it("never prompts for a Vercel login on a direct route without Vercel credentials (#164)", async () => { + const verifyModelRoute = vi.fn(async () => undefined); + const resolveRoute = vi.fn(async () => resolved()); + const ensureLink = vi.fn(async () => { + const error = new Error("Vercel authentication required"); + Object.assign(error, { code: "VERCEL_AUTH_REQUIRED" }); + throw error; + }); + const env: Record = { OPENAI_API_KEY: "secret" }; + const result = await ensureConnectedWorkspace({ + workspaceDir: "/workspace", + interactive: true, + modelRoute: route, + agentTypes: ["codex"], + env, + dependencies: { + ensureLink, + resolveRoute, + verifyModelRoute, + now: () => new Date("2026-01-01T00:00:00Z"), + }, + }); + // Attempted a silent (non-interactive) link, never a prompted login. + expect(ensureLink).toHaveBeenCalledWith(expect.objectContaining({ interactive: false })); + expect(resolveRoute).toHaveBeenCalledOnce(); + expect(verifyModelRoute).toHaveBeenCalledOnce(); + expect(env.OPENAI_API_KEY).toBe("secret"); + expect(result.project).toBeUndefined(); + expect(result.platformAuth).toBeUndefined(); + expect(result.sandboxReady).toBe(false); + expect(result.modelRouteVerified).toBe(true); + expect(result.verification.project).toBeUndefined(); + }); + + it("reuses an existing Vercel link silently on a direct route with credentials", async () => { + const verifyModelRoute = vi.fn(async () => undefined); + const resolveRoute = vi.fn(async () => resolved()); + const env: Record = { + OPENAI_API_KEY: "secret", + VERCEL_TOKEN: "v", + VERCEL_TEAM_ID: "team", + VERCEL_PROJECT_ID: "project", + }; + const result = await ensureConnectedWorkspace({ + workspaceDir: "/workspace", + interactive: true, + modelRoute: route, + agentTypes: ["codex"], + env, + dependencies: { + ensureLink: async () => ({ + method: "access-token-triple" as const, + project: { teamId: "team", projectId: "project" }, + link: { orgId: "team", projectId: "project" }, + }), + resolveRoute, + verifyModelRoute, + now: () => new Date("2026-01-01T00:00:00Z"), + }, + }); + expect(resolveRoute).toHaveBeenCalledOnce(); + expect(result.project).toEqual({ teamId: "team", projectId: "project" }); + expect(result.sandboxReady).toBe(true); + expect(result.verification.project).toEqual({ teamId: "team", projectId: "project" }); + }); + + it("rethrows link failures other than missing Vercel auth on a direct route", async () => { + const ensureLink = vi.fn(async () => { + throw new Error("network unreachable"); + }); + await expect( + ensureConnectedWorkspace({ + workspaceDir: "/workspace", + interactive: true, + modelRoute: route, + agentTypes: ["codex"], + env: { OPENAI_API_KEY: "secret" }, + dependencies: { + ensureLink, + resolveRoute: vi.fn(async () => resolved()), + verifyModelRoute: vi.fn(async () => undefined), + now: () => new Date("2026-01-01T00:00:00Z"), + }, + }), + ).rejects.toThrow("network unreachable"); + }); }); diff --git a/packages/deepsec/src/__tests__/export-triage.test.ts b/packages/deepsec/src/__tests__/export-triage.test.ts new file mode 100644 index 00000000..85952d4c --- /dev/null +++ b/packages/deepsec/src/__tests__/export-triage.test.ts @@ -0,0 +1,142 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import type { FileRecord } from "@deepsec/core"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { exportCommand } from "../commands/export.js"; + +function makeRecord(overrides: Partial = {}): FileRecord { + return { + filePath: "src/foo.ts", + projectId: "test-proj", + candidates: [], + lastScannedAt: "2026-05-08T00:00:00.000Z", + lastScannedRunId: "scan1", + fileHash: "h", + findings: [ + { + severity: "HIGH", + vulnSlug: "xss", + title: "Reflected XSS in handler", + description: "User input flows to response without escaping.", + lineNumbers: [42], + recommendation: "Escape the value.", + confidence: "high", + triage: { + priority: "P0", + exploitability: "trivial", + impact: "high", + reasoning: "External attacker crafts a malicious URL.", + triagedAt: "2026-05-08T12:00:00.000Z", + model: "claude-sonnet-4-6", + }, + }, + ], + analysisHistory: [ + { + runId: "run-1", + investigatedAt: "2026-05-08T11:00:00.000Z", + durationMs: 1000, + agentType: "claude-agent-sdk", + model: "claude-opus-4-7", + modelConfig: {}, + findingCount: 1, + }, + ], + status: "analyzed", + ...overrides, + }; +} + +let prevDataRoot: string | undefined; +let tmpRoot: string; +let logSpy: ReturnType; + +beforeEach(() => { + tmpRoot = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-export-triage-")); + prevDataRoot = process.env.DEEPSEC_DATA_ROOT; + process.env.DEEPSEC_DATA_ROOT = tmpRoot; + + const projectId = "test-proj"; + const projectDir = path.join(tmpRoot, projectId); + fs.mkdirSync(projectDir, { recursive: true }); + fs.writeFileSync( + path.join(projectDir, "project.json"), + JSON.stringify({ + projectId, + rootPath: "/tmp/fake", + createdAt: "2026-05-08T00:00:00.000Z", + }), + ); + + const record = makeRecord(); + const recordPath = path.join(projectDir, "files", record.filePath + ".json"); + fs.mkdirSync(path.dirname(recordPath), { recursive: true }); + fs.writeFileSync(recordPath, JSON.stringify(record)); + + logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); +}); + +afterEach(() => { + logSpy.mockRestore(); + fs.rmSync(tmpRoot, { recursive: true, force: true }); + if (prevDataRoot === undefined) delete process.env.DEEPSEC_DATA_ROOT; + else process.env.DEEPSEC_DATA_ROOT = prevDataRoot; +}); + +describe("export propagates triage data (issue #64)", () => { + it("includes metadata.triage in the JSON export", async () => { + const out = path.join(tmpRoot, "findings.json"); + await exportCommand({ projectId: "test-proj", format: "json", out }); + + const data = JSON.parse(fs.readFileSync(out, "utf-8")); + expect(data).toHaveLength(1); + expect(data[0].metadata.triage).toEqual({ + priority: "P0", + exploitability: "trivial", + impact: "high", + reasoning: "External attacker crafts a malicious URL.", + triagedAt: "2026-05-08T12:00:00.000Z", + model: "claude-sonnet-4-6", + }); + }); + + it("renders triage fields in the md-dir export body", async () => { + const outDir = path.join(tmpRoot, "md-out"); + await exportCommand({ projectId: "test-proj", format: "md-dir", out: outDir }); + + const sevDir = path.join(outDir, "HIGH"); + const files = fs.readdirSync(sevDir).filter((n) => n.endsWith(".md")); + expect(files).toHaveLength(1); + + const body = fs.readFileSync(path.join(sevDir, files[0]), "utf-8"); + expect(body).toContain("## Triage"); + expect(body).toContain("**Priority:** P0"); + expect(body).toContain("**Exploitability:** trivial"); + expect(body).toContain("**Impact:** high"); + expect(body).toContain("External attacker crafts a malicious URL."); + expect(body).toContain("2026-05-08T12:00:00.000Z"); + expect(body).toContain("claude-sonnet-4-6"); + }); + + it("omits metadata.triage and the Triage section when no triage is present", async () => { + const projectDir = path.join(tmpRoot, "test-proj"); + const record = makeRecord(); + record.findings[0].triage = undefined; + fs.writeFileSync( + path.join(projectDir, "files", record.filePath + ".json"), + JSON.stringify(record), + ); + + const jsonOut = path.join(tmpRoot, "findings-no-triage.json"); + await exportCommand({ projectId: "test-proj", format: "json", out: jsonOut }); + const data = JSON.parse(fs.readFileSync(jsonOut, "utf-8")); + expect(data[0].metadata.triage).toBeUndefined(); + + const mdOut = path.join(tmpRoot, "md-out-no-triage"); + await exportCommand({ projectId: "test-proj", format: "md-dir", out: mdOut }); + const files = fs.readdirSync(path.join(mdOut, "HIGH")).filter((n) => n.endsWith(".md")); + const body = fs.readFileSync(path.join(mdOut, "HIGH", files[0]), "utf-8"); + expect(body).not.toContain("## Triage"); + }); +}); diff --git a/packages/deepsec/src/__tests__/metrics.test.ts b/packages/deepsec/src/__tests__/metrics.test.ts index 5d8c5ded..b413228b 100644 --- a/packages/deepsec/src/__tests__/metrics.test.ts +++ b/packages/deepsec/src/__tests__/metrics.test.ts @@ -33,6 +33,11 @@ describe("formatTokens", () => { expect(formatTokens(950)).toBe("950"); }); + it("rounds floats under 1K to whole numbers", () => { + expect(formatTokens(12.7)).toBe("13"); + expect(formatTokens(12.3)).toBe("12"); + }); + it("uses K/M/B suffixes", () => { expect(formatTokens(1500)).toBe("1.5K"); expect(formatTokens(2_500_000)).toBe("2.5M"); diff --git a/packages/deepsec/src/__tests__/network-policy.test.ts b/packages/deepsec/src/__tests__/network-policy.test.ts index 2ff6a3f9..053136ae 100644 --- a/packages/deepsec/src/__tests__/network-policy.test.ts +++ b/packages/deepsec/src/__tests__/network-policy.test.ts @@ -77,6 +77,12 @@ describe("buildWorkerNetworkPolicy", () => { const piPolicy = buildWorkerNetworkPolicy({}, "pi"); expect(allowedHosts(piPolicy)).toEqual(["ai-gateway.vercel.sh"]); + + const openCodePolicy = buildWorkerNetworkPolicy( + { DEEPSEC_OPENCODE_PROVIDER: "anthropic" }, + "opencode", + ); + expect(allowedHosts(openCodePolicy)).toEqual(["api.anthropic.com"]); }); it("falls back when the URL is unparseable", () => { @@ -141,6 +147,45 @@ describe("buildWorkerNetworkPolicy", () => { expect(bearerFor(policy, "api.withmartian.com")).toBe("Bearer martian-real"); }); + it("uses the Anthropic route and token for the default OpenCode model", () => { + const policy = buildWorkerNetworkPolicy( + { + DEEPSEC_OPENCODE_PROVIDER: "anthropic", + ANTHROPIC_UPSTREAM_BASE_URL: "https://ai-gateway.vercel.sh", + }, + "opencode", + { anthropicToken: "vck_opencode" }, + ); + expect(allowedHosts(policy)).toEqual(["ai-gateway.vercel.sh"]); + expect(bearerFor(policy, "ai-gateway.vercel.sh")).toBe("Bearer vck_opencode"); + }); + + it("uses the OpenAI route and token for an OpenCode OpenAI model", () => { + const policy = buildWorkerNetworkPolicy( + { + DEEPSEC_OPENCODE_PROVIDER: "openai", + OPENAI_BASE_URL: "https://api.openai.com/v1", + }, + "opencode", + { openaiToken: "sk-opencode" }, + ); + expect(allowedHosts(policy)).toEqual(["api.openai.com"]); + expect(bearerFor(policy, "api.openai.com")).toBe("Bearer sk-opencode"); + }); + + it("uses a custom OpenCode provider host and brokered token", () => { + const policy = buildWorkerNetworkPolicy( + { + DEEPSEC_OPENCODE_PROVIDER: "martian", + DEEPSEC_OPENCODE_AI_BASE_URL: "https://api.withmartian.com/v1", + }, + "opencode", + { customToken: { envName: "MARTIAN_API_KEY", token: "martian-real" } }, + ); + expect(allowedHosts(policy)).toEqual(["api.withmartian.com"]); + expect(bearerFor(policy, "api.withmartian.com")).toBe("Bearer martian-real"); + }); + it("emits an attribution-only transform when no matching credential is provided", () => { const policy = buildWorkerNetworkPolicy( { ANTHROPIC_UPSTREAM_BASE_URL: "https://ai-gateway.vercel.sh" }, diff --git a/packages/deepsec/src/__tests__/opencode-sandbox.test.ts b/packages/deepsec/src/__tests__/opencode-sandbox.test.ts new file mode 100644 index 00000000..a6eaaabe --- /dev/null +++ b/packages/deepsec/src/__tests__/opencode-sandbox.test.ts @@ -0,0 +1,47 @@ +import type { Sandbox } from "@vercel/sandbox"; +import { describe, expect, it, vi } from "vitest"; +import { DEEPSEC_DIR, ensureOpenCodeBinary } from "../sandbox/setup.js"; + +function commandResult(exitCode: number, stdout = "", stderr = "") { + return { + exitCode, + stdout: async () => stdout, + stderr: async () => stderr, + }; +} + +describe("ensureOpenCodeBinary", () => { + it.each([ + ["dev", `${DEEPSEC_DIR}/packages/deepsec/node_modules/opencode-ai/bin/opencode.exe`], + ["installed", `${DEEPSEC_DIR}/node_modules/deepsec/node_modules/opencode-ai/bin/opencode.exe`], + ] as const)("links the %s package binary onto PATH", async (mode, source) => { + const runCommand = vi + .fn() + .mockResolvedValueOnce(commandResult(0)) + .mockResolvedValueOnce(commandResult(0, "1.18.3\n")); + const onLog = vi.fn(); + + await ensureOpenCodeBinary({ runCommand } as unknown as Sandbox, mode, onLog); + + expect(runCommand).toHaveBeenNthCalledWith(1, { + cmd: "ln", + args: ["-sfn", source, "/usr/local/bin/opencode"], + sudo: true, + }); + expect(runCommand).toHaveBeenNthCalledWith(2, { + cmd: "opencode", + args: ["--version"], + cwd: DEEPSEC_DIR, + }); + expect(onLog).toHaveBeenCalledWith(" 1.18.3"); + }); + + it("reports a failed system link before checking the binary", async () => { + const runCommand = vi.fn().mockResolvedValue(commandResult(1, "", "permission denied")); + + await expect( + ensureOpenCodeBinary({ runCommand } as unknown as Sandbox, "dev", vi.fn()), + ).rejects.toThrow("OpenCode native binary link failed (exit 1): permission denied"); + expect(runCommand).toHaveBeenCalledTimes(1); + }); +}); diff --git a/packages/deepsec/src/__tests__/preflight.test.ts b/packages/deepsec/src/__tests__/preflight.test.ts index de65597f..264e9ad7 100644 --- a/packages/deepsec/src/__tests__/preflight.test.ts +++ b/packages/deepsec/src/__tests__/preflight.test.ts @@ -85,6 +85,7 @@ describe("assertAgentCredential", () => { CLAUDE_HOME: process.env.CLAUDE_HOME, CODEX_HOME: process.env.CODEX_HOME, PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR, + XDG_DATA_HOME: process.env.XDG_DATA_HOME, PATH: process.env.PATH, }; delete process.env.ANTHROPIC_AUTH_TOKEN; @@ -102,7 +103,9 @@ describe("assertAgentCredential", () => { process.env.CLAUDE_HOME = emptyClaudeHome; process.env.CODEX_HOME = emptyCodexHome; process.env.PI_CODING_AGENT_DIR = join(emptyCodexHome, "pi-agent"); + process.env.XDG_DATA_HOME = join(emptyCodexHome, "xdg-data"); mkdirSync(process.env.PI_CODING_AGENT_DIR, { recursive: true }); + mkdirSync(process.env.XDG_DATA_HOME, { recursive: true }); process.env.PATH = emptyPathDir; }); afterEach(() => { @@ -221,6 +224,33 @@ describe("assertAgentCredential", () => { writeFileSync(join(piHome, "auth.json"), "{}"); expect(() => assertAgentCredential("pi", { inSandbox: true })).toThrow(/AI_GATEWAY_API_KEY/); }); + + it("passes for opencode when AI Gateway credentials are set", () => { + process.env.AI_GATEWAY_API_KEY = "vck_gateway"; + expect(() => assertAgentCredential("opencode")).not.toThrow(); + }); + + it("passes for opencode when local provider auth exists", () => { + const authDir = join(process.env.XDG_DATA_HOME!, "opencode"); + mkdirSync(authDir, { recursive: true }); + writeFileSync(join(authDir, "auth.json"), "{}"); + expect(() => assertAgentCredential("opencode")).not.toThrow(); + }); + + it("ignores local OpenCode auth in sandbox mode", () => { + const authDir = join(process.env.XDG_DATA_HOME!, "opencode"); + mkdirSync(authDir, { recursive: true }); + writeFileSync(join(authDir, "auth.json"), "{}"); + expect(() => assertAgentCredential("opencode")).not.toThrow(); + expect(() => assertAgentCredential("opencode", { inSandbox: true })).toThrow( + /AI_GATEWAY_API_KEY/, + ); + }); + + it("shows OpenCode connection instructions when credentials are missing", () => { + expect(() => assertAgentCredential("opencode")).toThrow(/--agent opencode/); + expect(() => assertAgentCredential("opencode")).toThrow(/\/connect/); + }); }); describe("assertSandboxCredential", () => { diff --git a/packages/deepsec/src/__tests__/resolve-agent-type.test.ts b/packages/deepsec/src/__tests__/resolve-agent-type.test.ts index d9daa6e8..8299f20d 100644 --- a/packages/deepsec/src/__tests__/resolve-agent-type.test.ts +++ b/packages/deepsec/src/__tests__/resolve-agent-type.test.ts @@ -20,6 +20,10 @@ describe("resolveAgentType", () => { expect(resolveAgentType("pi")).toBe("pi"); }); + it("accepts opencode as a built-in agent type", () => { + expect(resolveAgentType("opencode")).toBe("opencode"); + }); + it("aliases claude from defaultAgent config", () => { setLoadedConfig(defineConfig({ projects: [], defaultAgent: "claude" })); expect(resolveAgentType(undefined)).toBe("claude-agent-sdk"); diff --git a/packages/deepsec/src/__tests__/setup-install.test.ts b/packages/deepsec/src/__tests__/setup-install.test.ts index bef1d955..c52ceca8 100644 --- a/packages/deepsec/src/__tests__/setup-install.test.ts +++ b/packages/deepsec/src/__tests__/setup-install.test.ts @@ -1,8 +1,8 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -import { describe, expect, it } from "vitest"; -import { selectPackageManager } from "../setup/install.js"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { probeWorkspaceInstall, selectPackageManager } from "../setup/install.js"; describe("setup package-manager selection", () => { it("falls back to npm when a preferred pnpm is unavailable", () => { @@ -21,3 +21,52 @@ describe("setup package-manager selection", () => { ); }); }); + +describe("post-install workspace probe", () => { + afterEach(() => { + vi.restoreAllMocks(); + }); + + function installedWorkspace(): string { + const workspace = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-install-")); + const installed = path.join(workspace, "node_modules", "deepsec", "dist"); + fs.mkdirSync(installed, { recursive: true }); + fs.writeFileSync( + path.join(workspace, "node_modules", "deepsec", "package.json"), + JSON.stringify({ version: "1.2.3" }), + ); + fs.writeFileSync(path.join(installed, "cli.mjs"), ""); + fs.writeFileSync(path.join(installed, "config.mjs"), ""); + return workspace; + } + + // The installer is a child process that just wrote node_modules, so on a + // non-coherent mount the probe that follows it can still see nothing there + // and report "workspace is unusable" for a perfectly good install. + function lagFirstLookup(): void { + let calls = 0; + const real = fs.existsSync; + vi.spyOn(fs, "existsSync").mockImplementation((target) => { + calls += 1; + return calls === 1 ? false : real(target); + }); + } + + it("tolerates a lagging lookup right after the installer exits", () => { + const workspace = installedWorkspace(); + vi.stubEnv("DEEPSEC_FS_SETTLE_MS", "0"); + lagFirstLookup(); + expect(probeWorkspaceInstall(workspace, { settle: true })).toEqual({ + ok: true, + version: "1.2.3", + }); + }); + + it("does not wait out a genuinely missing install", () => { + const workspace = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-install-")); + expect(probeWorkspaceInstall(workspace)).toEqual({ + ok: false, + reason: "node_modules/deepsec is missing", + }); + }); +}); diff --git a/packages/deepsec/src/__tests__/tarball.test.ts b/packages/deepsec/src/__tests__/tarball.test.ts index 3a3c63d8..9bd999f0 100644 --- a/packages/deepsec/src/__tests__/tarball.test.ts +++ b/packages/deepsec/src/__tests__/tarball.test.ts @@ -223,7 +223,11 @@ describe("extractTarballLocally — strict allowlist", () => { expect("files/../etc/passwd.json".split("/").includes("..")).toBe(true); }); - it("skips an oversized entry but extracts the rest of the tarball", async () => { + // The 64MiB random-byte fixture plus tar+extract trips vitest's default + // 5s timeout on slower machines and under parallel collection (#51). + it("skips an oversized entry but extracts the rest of the tarball", { + timeout: 30_000, + }, async () => { const projectId = path.basename(destDir); const src = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-tar-fat-")); fs.mkdirSync(path.join(src, "files")); diff --git a/packages/deepsec/src/agent-config.ts b/packages/deepsec/src/agent-config.ts index 8379ca7b..a627cf15 100644 --- a/packages/deepsec/src/agent-config.ts +++ b/packages/deepsec/src/agent-config.ts @@ -51,7 +51,7 @@ export function buildAgentConfig(opts: AgentRuntimeOpts): Record = { @@ -65,8 +65,8 @@ export function buildAgentConfig(opts: AgentRuntimeOpts): Record", "Resume a specific processing run") .option( "--agent ", - "Agent plugin type: codex, claude, or pi (default: defaultAgent in deepsec.config.ts, else codex)", + "Agent plugin type: codex, claude, opencode, or pi (default: defaultAgent in deepsec.config.ts, else codex)", ) .option( "--model ", - "Model to use (default: claude-opus-4-8 for claude, gpt-5.5 for codex, zai/glm-5.2 for pi)", + "Model to use (default: claude-opus-4-8 for claude, gpt-5.5 for codex, anthropic/claude-opus-4-8 for opencode, zai/glm-5.2 for pi)", ) .option( "--ai-provider ", - "Pi: provider to override for --ai-base-url / --ai-api-key-env (e.g. openai)", + "Pi/OpenCode: provider to override for --ai-base-url / --ai-api-key-env (e.g. openai)", ) .option( "--ai-base-url ", - "Pi: provider base URL override (e.g. a Martian/OpenAI-compatible gateway)", + "Pi/OpenCode: provider base URL override (e.g. a Martian/OpenAI-compatible gateway)", + ) + .option( + "--ai-api-key-env ", + "Pi/OpenCode: environment variable that holds the provider API key", ) - .option("--ai-api-key-env ", "Pi: environment variable that holds the provider API key") .option( "--ai-header ", - "Pi: extra provider request header; repeatable", + "Pi/OpenCode: extra provider request header; repeatable", collectRepeatable, [], ) @@ -370,24 +373,27 @@ program .option("--run-id ", "Resume a specific revalidation run") .option( "--agent ", - "Agent plugin type: codex, claude, or pi (default: defaultAgent in deepsec.config.ts, else codex)", + "Agent plugin type: codex, claude, opencode, or pi (default: defaultAgent in deepsec.config.ts, else codex)", ) .option( "--model ", - "Model to use (default: claude-opus-4-8 for claude, gpt-5.5 for codex, zai/glm-5.2 for pi)", + "Model to use (default: claude-opus-4-8 for claude, gpt-5.5 for codex, anthropic/claude-opus-4-8 for opencode, zai/glm-5.2 for pi)", ) .option( "--ai-provider ", - "Pi: provider to override for --ai-base-url / --ai-api-key-env (e.g. openai)", + "Pi/OpenCode: provider to override for --ai-base-url / --ai-api-key-env (e.g. openai)", ) .option( "--ai-base-url ", - "Pi: provider base URL override (e.g. a Martian/OpenAI-compatible gateway)", + "Pi/OpenCode: provider base URL override (e.g. a Martian/OpenAI-compatible gateway)", + ) + .option( + "--ai-api-key-env ", + "Pi/OpenCode: environment variable that holds the provider API key", ) - .option("--ai-api-key-env ", "Pi: environment variable that holds the provider API key") .option( "--ai-header ", - "Pi: extra provider request header; repeatable", + "Pi/OpenCode: extra provider request header; repeatable", collectRepeatable, [], ) @@ -482,7 +488,7 @@ program .option("--require-owner", "Drop findings that have no ownership data (no assignee, no teams)") .option( "--only-agent ", - "Only export findings produced by this agent backend (e.g. codex, claude, pi)", + "Only export findings produced by this agent backend (e.g. codex, claude, opencode, pi)", ) .option( "--only-marker ", diff --git a/packages/deepsec/src/commands/export.ts b/packages/deepsec/src/commands/export.ts index 3ed50a7f..a9ae2774 100644 --- a/packages/deepsec/src/commands/export.ts +++ b/packages/deepsec/src/commands/export.ts @@ -1,20 +1,11 @@ import crypto from "node:crypto"; import fs from "node:fs"; import path from "node:path"; -import type { FileRecord, Finding, Severity } from "@deepsec/core"; -import { dataDir, getDataRoot, loadAllFileRecords } from "@deepsec/core"; +import type { FileRecord, Finding, Severity, Triage } from "@deepsec/core"; +import { dataDir, getDataRoot, loadAllFileRecords, SEVERITY_ORDER } from "@deepsec/core"; import { BOLD, DIM, GREEN, RESET, YELLOW } from "../formatters.js"; import { resolveAgentType } from "../resolve-agent-type.js"; -const SEVERITY_ORDER: Record = { - CRITICAL: 0, - HIGH: 1, - HIGH_BUG: 2, - MEDIUM: 3, - BUG: 4, - LOW: 5, -}; - interface OwnerSummary { assignee?: string; assigneeSource?: "oncall" | "manager" | "top-contributor" | "last-committer"; @@ -45,6 +36,7 @@ interface ExportedFinding { verdict: string; reasoning: string; }; + triage?: Triage; githubUrl?: string; owners: OwnerSummary; }; @@ -167,6 +159,19 @@ function buildDescription( } } + if (finding.triage) { + parts.push( + "", + "## Triage", + "", + `**Priority:** ${finding.triage.priority} • **Exploitability:** ${finding.triage.exploitability} • **Impact:** ${finding.triage.impact}`, + "", + finding.triage.reasoning, + "", + `_Triaged ${finding.triage.triagedAt} by ${finding.triage.model}._`, + ); + } + parts.push( "", "## Finding", @@ -512,6 +517,7 @@ export async function exportCommand(opts: { revalidation: finding.revalidation ? { verdict: finding.revalidation.verdict, reasoning: finding.revalidation.reasoning } : undefined, + triage: finding.triage, githubUrl, owners, }, diff --git a/packages/deepsec/src/commands/init-project.ts b/packages/deepsec/src/commands/init-project.ts index d50777b7..136792ec 100644 --- a/packages/deepsec/src/commands/init-project.ts +++ b/packages/deepsec/src/commands/init-project.ts @@ -1,6 +1,6 @@ import fs from "node:fs"; import path from "node:path"; -import { dataDir, ensureProject } from "@deepsec/core"; +import { dataDir, ensureProject, mkdirSettled } from "@deepsec/core"; import { BOLD, CYAN, DIM, GREEN, RESET, YELLOW } from "../formatters.js"; import { requireExistingDir } from "../require-dir.js"; import { validateProjectId } from "../resolve-project-id.js"; @@ -90,7 +90,7 @@ export function registerProject(opts: { process.chdir(workspaceDir); ensureProject(id, targetAbs); const projectDir = dataDir(id); - fs.mkdirSync(projectDir, { recursive: true }); + mkdirSettled(projectDir); const infoMdPath = path.join(projectDir, "INFO.md"); if (!fs.existsSync(infoMdPath) || opts.force) { fs.writeFileSync(infoMdPath, infoMdTemplate(id)); diff --git a/packages/deepsec/src/commands/init.ts b/packages/deepsec/src/commands/init.ts index 06053386..d94cbd0b 100644 --- a/packages/deepsec/src/commands/init.ts +++ b/packages/deepsec/src/commands/init.ts @@ -1,6 +1,6 @@ import fs from "node:fs"; import path from "node:path"; -import { ensureProject } from "@deepsec/core"; +import { ensureProject, mkdirSettled } from "@deepsec/core"; import { type ModelProfile, parseModelProfile, @@ -159,7 +159,7 @@ export async function initCommand(opts: InitOpts) { } // Workspace skeleton: empty config (with marker), README, AGENTS, env. - fs.mkdirSync(workspaceDir, { recursive: true }); + mkdirSettled(workspaceDir); writeFile( workspaceDir, "package.json", @@ -267,12 +267,16 @@ export async function initCommand(opts: InitOpts) { // avoids transferring stdin between two independent prompt runtimes. // // A local-subscription route needs no platform link, so skip it rather - // than prompt for the Vercel login the user just declined. On resume the - // route comes from the login checkpoint, or from the generated config when - // setup was interrupted before login; the prompt above is first-run only. + // than prompt for the Vercel login the user just declined. Direct and + // custom routes likewise carry their own provider credential — the + // platform link only enables the optional Vercel Sandbox, so it is set + // up silently later if credentials exist, never prompted here (#164). + // On resume the route comes from the login checkpoint, or from the + // generated config when setup was interrupted before login; the prompt + // above is first-run only. const linkRoute = modelRoute ?? persistedModelRoute(workspaceDir, registered.id); if ( - linkRoute?.mode !== "local" && + (linkRoute?.mode ?? "gateway") === "gateway" && !headless && process.stdin.isTTY && process.stdout.isTTY && diff --git a/packages/deepsec/src/commands/metrics.ts b/packages/deepsec/src/commands/metrics.ts index 8cc8683e..208c9052 100644 --- a/packages/deepsec/src/commands/metrics.ts +++ b/packages/deepsec/src/commands/metrics.ts @@ -160,7 +160,7 @@ export function formatCost(n: number): string { export function formatTokens(n: number): string { if (n === 0) return "0"; - if (n < 1000) return String(n); + if (n < 1000) return String(Math.round(n)); if (n < 1_000_000) return `${(n / 1000).toFixed(1)}K`; if (n < 1_000_000_000) return `${(n / 1_000_000).toFixed(1)}M`; return `${(n / 1_000_000_000).toFixed(1)}B`; @@ -409,8 +409,8 @@ export function metricsCommand(opts: { projectId?: string; minSeverity?: string m.projectId, String(m.analysisCount), formatCost(m.cost), - formatTokens(m.tokens.input), - formatTokens(m.tokens.output), + formatTokens(Math.round(m.tokens.input)), + formatTokens(Math.round(m.tokens.output)), cacheHit, formatCost(perAnalysis), ], @@ -430,8 +430,8 @@ export function metricsCommand(opts: { projectId?: string; minSeverity?: string `${BOLD}TOTAL${RESET}`, `${BOLD}${totals.analysisCount}${RESET}`, `${BOLD}${formatCost(totals.cost)}${RESET}`, - `${BOLD}${formatTokens(totals.tokens.input)}${RESET}`, - `${BOLD}${formatTokens(totals.tokens.output)}${RESET}`, + `${BOLD}${formatTokens(Math.round(totals.tokens.input))}${RESET}`, + `${BOLD}${formatTokens(Math.round(totals.tokens.output))}${RESET}`, `${BOLD}${cacheHit}${RESET}`, `${BOLD}${formatCost(perAnalysis)}${RESET}`, ], diff --git a/packages/deepsec/src/commands/sandbox-all.ts b/packages/deepsec/src/commands/sandbox-all.ts index b0a3a974..9af63493 100644 --- a/packages/deepsec/src/commands/sandbox-all.ts +++ b/packages/deepsec/src/commands/sandbox-all.ts @@ -216,6 +216,7 @@ export async function sandboxAllCommand( agentType, aiApiKeyEnv: explicitAiApiKeyEnv, aiBaseUrl: explicitAiBaseUrl ?? resolvedRoute?.route.baseUrl, + aiProvider: extractFlag(passthrough, "--ai-provider"), brokeredModelCredential: resolvedRoute?.broker, model: extractFlag(passthrough, "--model") ?? defaultModelForAgent(agentType), snapshotId: undefined, diff --git a/packages/deepsec/src/commands/sandbox-process.ts b/packages/deepsec/src/commands/sandbox-process.ts index b305be0c..ad335ca0 100644 --- a/packages/deepsec/src/commands/sandbox-process.ts +++ b/packages/deepsec/src/commands/sandbox-process.ts @@ -80,6 +80,7 @@ function buildConfig( agentType, aiApiKeyEnv: extractFlag(args, "--ai-api-key-env"), aiBaseUrl: extractFlag(args, "--ai-base-url"), + aiProvider: extractFlag(args, "--ai-provider"), model: extractFlag(args, "--model") ?? defaultModelForAgent(agentType), snapshotId: opts.snapshotId, saveSnapshot: opts.saveSnapshot ?? false, diff --git a/packages/deepsec/src/pr-comment.ts b/packages/deepsec/src/pr-comment.ts index 01ef990a..363a6df3 100644 --- a/packages/deepsec/src/pr-comment.ts +++ b/packages/deepsec/src/pr-comment.ts @@ -1,18 +1,4 @@ -import { type FileRecord, loadAllFileRecords, type Severity } from "@deepsec/core"; - -/** - * Severity ordering used to sort findings within the PR comment. Mirrors - * the order in `packages/processor/src/index.ts:SEVERITY_ORDER` — keep them - * in sync if you add a tier. - */ -const SEVERITY_ORDER: Record = { - CRITICAL: 0, - HIGH: 1, - MEDIUM: 2, - HIGH_BUG: 3, - BUG: 4, - LOW: 5, -}; +import { type FileRecord, loadAllFileRecords, SEVERITY_ORDER, type Severity } from "@deepsec/core"; const SEVERITY_BADGE: Record = { CRITICAL: "🔴 CRITICAL", diff --git a/packages/deepsec/src/preflight.ts b/packages/deepsec/src/preflight.ts index af6e8daf..72a374e5 100644 --- a/packages/deepsec/src/preflight.ts +++ b/packages/deepsec/src/preflight.ts @@ -169,6 +169,10 @@ function isPi(agentType: string | undefined): boolean { return agentType === "pi"; } +function isOpenCode(agentType: string | undefined): boolean { + return agentType === "opencode"; +} + /** * Walk `$PATH` looking for a binary. Used as a positive signal that an * agent CLI (`claude`, `codex`) is set up on this host — if it's @@ -227,11 +231,21 @@ function hasLocalPiAgent(): boolean { return existsSync(join(piHome, "auth.json")); } +/** + * OpenCode stores provider credentials in its XDG data directory. The CLI + * binary is a deepsec runtime dependency, so merely finding `opencode` on + * PATH is not evidence that the user connected a provider. + */ +function hasLocalOpenCodeAuth(): boolean { + const dataHome = process.env.XDG_DATA_HOME || join(homedir(), ".local", "share"); + return existsSync(join(dataHome, "opencode", "auth.json")); +} + // Built-in backends we know how to credential-check. Agents registered // via plugins (deepsec.config.ts → plugins: [{ agents: [...] }]) handle // their own credential resolution, so we skip the check for anything // other than these. -const KNOWN_BACKENDS = new Set(["claude-agent-sdk", "codex", "pi"]); +const KNOWN_BACKENDS = new Set(["claude-agent-sdk", "codex", "opencode", "pi"]); /** * Verify the orchestrator has an AI credential the chosen agent can use. @@ -311,6 +325,21 @@ export function assertAgentCredential( ); } + if (isOpenCode(agentType)) { + if (gateway || custom || anthropic || anthropicApi || openai) return; + if (!options.inSandbox && hasLocalOpenCodeAuth()) return; + const customHint = options.aiApiKeyEnv + ? `, or set ${options.aiApiKeyEnv}=… for the selected --ai-api-key-env` + : ""; + throw new Error( + `Missing AI credentials for --agent opencode.\n` + + `\n` + + ` Add to .env.local: AI_GATEWAY_API_KEY=vck_…${customHint}\n` + + ` Or run opencode and use /connect to authenticate a provider (local runs only).\n` + + ` Setup: ${SETUP_DOC_URL}`, + ); + } + if (anthropic || anthropicApi) return; if (!options.inSandbox && hasLocalClaudeAgent()) return; const displayAgent = diff --git a/packages/deepsec/src/sandbox/orchestrator.ts b/packages/deepsec/src/sandbox/orchestrator.ts index 1dd2dd79..e52c8dfd 100644 --- a/packages/deepsec/src/sandbox/orchestrator.ts +++ b/packages/deepsec/src/sandbox/orchestrator.ts @@ -354,6 +354,8 @@ async function bootstrapAndSpawn( agentType: config.agentType, aiApiKeyEnv: config.aiApiKeyEnv, aiBaseUrl: config.aiBaseUrl, + aiProvider: config.aiProvider, + model: config.model, brokeredModelCredential: config.brokeredModelCredential, vcpus: config.vcpus, timeout: config.timeout, diff --git a/packages/deepsec/src/sandbox/partitioner.ts b/packages/deepsec/src/sandbox/partitioner.ts index ed803982..8e0a027d 100644 --- a/packages/deepsec/src/sandbox/partitioner.ts +++ b/packages/deepsec/src/sandbox/partitioner.ts @@ -1,17 +1,13 @@ import fs from "node:fs"; import path from "node:path"; import type { FileRecord } from "@deepsec/core"; -import { dataDir, loadAllFileRecords } from "@deepsec/core"; +import { dataDir, loadAllFileRecords, SEVERITY_ORDER } from "@deepsec/core"; import { noiseScore } from "@deepsec/scanner"; import type { PartitionResult, SandboxSubcommand } from "./types.js"; -const SEVERITY_ORDER: Record = { - CRITICAL: 0, - HIGH: 1, - MEDIUM: 2, - HIGH_BUG: 3, - BUG: 4, -}; +function severityRank(severity: string): number { + return SEVERITY_ORDER[severity as keyof typeof SEVERITY_ORDER] ?? 99; +} /** * Load eligible files for the given command and split into N disjoint partitions. @@ -68,7 +64,7 @@ export function partitionFiles( if (r.findings.length === 0) return false; const unrevalidated = r.findings.filter((f) => { if (!opts.force && f.revalidation) return false; - if (opts.minSeverity && SEVERITY_ORDER[f.severity] > SEVERITY_ORDER[opts.minSeverity]) + if (opts.minSeverity && severityRank(f.severity) > severityRank(opts.minSeverity)) return false; return true; }); @@ -95,8 +91,8 @@ export function partitionFiles( eligible.sort((a, b) => { if (command === "revalidate") { // Sort by severity (CRITICAL first) - const aBest = Math.min(...a.findings.map((f) => SEVERITY_ORDER[f.severity] ?? 99)); - const bBest = Math.min(...b.findings.map((f) => SEVERITY_ORDER[f.severity] ?? 99)); + const aBest = Math.min(...a.findings.map((f) => severityRank(f.severity))); + const bBest = Math.min(...b.findings.map((f) => severityRank(f.severity))); if (aBest !== bBest) return aBest - bBest; } diff --git a/packages/deepsec/src/sandbox/setup.ts b/packages/deepsec/src/sandbox/setup.ts index ebc7e975..8f20acfb 100644 --- a/packages/deepsec/src/sandbox/setup.ts +++ b/packages/deepsec/src/sandbox/setup.ts @@ -76,6 +76,8 @@ export interface BrokeredCredentials { export interface BrokeredCredentialOptions { aiApiKeyEnv?: string; aiBaseUrl?: string; + aiProvider?: string; + model?: string; brokeredModelCredential?: BrokeredModelCredential; } @@ -92,11 +94,12 @@ export function resolveBrokeredCredentials( if (options.brokeredModelCredential) { return { selected: options.brokeredModelCredential }; } - const anthropicToken = process.env.ANTHROPIC_AUTH_TOKEN; + const anthropicToken = process.env.ANTHROPIC_AUTH_TOKEN ?? process.env.ANTHROPIC_API_KEY; const explicitOpenai = process.env.OPENAI_API_KEY; + const configurableAgent = agentType === "pi" || agentType === "opencode"; const aiGatewayToken = agentType === "pi" ? process.env.AI_GATEWAY_API_KEY : undefined; const customToken = - agentType === "pi" && options.aiApiKeyEnv && process.env[options.aiApiKeyEnv] + configurableAgent && options.aiApiKeyEnv && process.env[options.aiApiKeyEnv] ? { envName: options.aiApiKeyEnv, token: process.env[options.aiApiKeyEnv]! } : undefined; // Only borrow ANTHROPIC for OPENAI on the codex path — and only when the @@ -117,7 +120,19 @@ const PROXY_SCRIPT_BY_MODE: Record = { dev: `${DEEPSEC_DIR}/packages/deepsec/src/sandbox/request-proxy.mjs`, installed: `${DEEPSEC_DIR}/node_modules/deepsec/dist/sandbox/request-proxy.mjs`, }; +const OPENCODE_BINARY_BY_MODE: Record = { + dev: `${DEEPSEC_DIR}/packages/deepsec/node_modules/opencode-ai/bin/opencode.exe`, + installed: `${DEEPSEC_DIR}/node_modules/deepsec/node_modules/opencode-ai/bin/opencode.exe`, +}; const CODEX_HOME = "/vercel/sandbox/.codex"; +const OPENCODE_PROVIDER_ENV = "DEEPSEC_OPENCODE_PROVIDER"; +const OPENCODE_CUSTOM_BASE_URL_ENV = "DEEPSEC_OPENCODE_AI_BASE_URL"; + +function providerFromModel(model: string | undefined): string | undefined { + if (!model) return undefined; + const slash = model.indexOf("/"); + return slash > 0 ? model.slice(0, slash) : undefined; +} export function buildSandboxEnv( agentType: string | undefined, @@ -162,6 +177,16 @@ export function buildSandboxEnv( env[PI_CUSTOM_BASE_URL_ENV] = options.aiBaseUrl; } } + if (agentType === "opencode") { + const provider = options.aiProvider ?? providerFromModel(options.model) ?? "anthropic"; + env[OPENCODE_PROVIDER_ENV] = provider; + if (credentials.customToken) { + env[credentials.customToken.envName] = BROKERED_TOKEN_PLACEHOLDER; + } + if (options.aiBaseUrl) { + env[OPENCODE_CUSTOM_BASE_URL_ENV] = options.aiBaseUrl; + } + } // Belt-and-suspenders alongside the worker egress firewall: the master // kill-switch covers DISABLE_TELEMETRY / DISABLE_ERROR_REPORTING / @@ -186,12 +211,13 @@ export function buildSandboxEnv( // body mutation needed for Codex, so a proxy hop would just add latency // and a base-url-rewriting hazard (path doubling, etc.). spawnFromSnapshot // skips the proxy startup when agentType=codex for the same reason. + const openCodeProvider = env[OPENCODE_PROVIDER_ENV]; if (agentType === "codex") { env["CODEX_HOME"] = CODEX_HOME; if (!env["OPENAI_BASE_URL"] && env["ANTHROPIC_BASE_URL"]) { env["OPENAI_BASE_URL"] = env["ANTHROPIC_BASE_URL"]; } - } else { + } else if (agentType !== "opencode" || openCodeProvider === "anthropic") { const realBaseUrl = env["ANTHROPIC_BASE_URL"]; if (realBaseUrl) { env["ANTHROPIC_UPSTREAM_BASE_URL"] = realBaseUrl; @@ -241,6 +267,9 @@ export function buildWorkerNetworkPolicy( ): NetworkPolicy { const isCodex = agentType === "codex"; const isPi = agentType === "pi"; + const isOpenCode = agentType === "opencode"; + const openCodeProvider = env[OPENCODE_PROVIDER_ENV] ?? "anthropic"; + const openCodeCustomBaseUrl = env[OPENCODE_CUSTOM_BASE_URL_ENV]; // Single AI host per backend. Prefer derived from the base URL the agent // will actually use; fall back to the provider's documented default when @@ -249,9 +278,13 @@ export function buildWorkerNetworkPolicy( credentials.selected?.host ?? (isPi ? (hostFromUrl(env[PI_CUSTOM_BASE_URL_ENV]) ?? DEFAULT_AI_GATEWAY_HOST) - : isCodex - ? (hostFromUrl(env["OPENAI_BASE_URL"]) ?? DEFAULT_OPENAI_HOST) - : (hostFromUrl(env["ANTHROPIC_UPSTREAM_BASE_URL"]) ?? DEFAULT_ANTHROPIC_HOST)); + : isOpenCode && openCodeCustomBaseUrl + ? (hostFromUrl(openCodeCustomBaseUrl) ?? DEFAULT_ANTHROPIC_HOST) + : isOpenCode && openCodeProvider === "openai" + ? (hostFromUrl(env["OPENAI_BASE_URL"]) ?? DEFAULT_OPENAI_HOST) + : isCodex + ? (hostFromUrl(env["OPENAI_BASE_URL"]) ?? DEFAULT_OPENAI_HOST) + : (hostFromUrl(env["ANTHROPIC_UPSTREAM_BASE_URL"]) ?? DEFAULT_ANTHROPIC_HOST)); // The fallback flips at resolveBrokeredCredentials — by here, openaiToken // already carries the ANTHROPIC gateway token if the user only set that @@ -261,9 +294,13 @@ export function buildWorkerNetworkPolicy( ? env[PI_CUSTOM_BASE_URL_ENV] ? credentials.customToken?.token : credentials.aiGatewayToken - : isCodex - ? credentials.openaiToken - : credentials.anthropicToken; + : isOpenCode && openCodeCustomBaseUrl + ? credentials.customToken?.token + : isOpenCode && openCodeProvider === "openai" + ? credentials.openaiToken + : isCodex + ? credentials.openaiToken + : credentials.anthropicToken; // App Attribution rides the same per-domain header rewrite that brokers // credentials, so every sandboxed agent's gateway traffic is tagged @@ -400,7 +437,7 @@ export async function createBootstrapSnapshot(opts: BootstrapOptions): Promise { const credentialOptions = { aiApiKeyEnv: opts.aiApiKeyEnv, aiBaseUrl: opts.aiBaseUrl, + aiProvider: opts.aiProvider, + model: opts.model, brokeredModelCredential: opts.brokeredModelCredential, }; const credentials = resolveBrokeredCredentials(opts.agentType, credentialOptions); @@ -505,7 +549,11 @@ export async function spawnFromSnapshot(opts: SpawnOptions): Promise { // keeps a stub agent out of the proxy startup, which fails fast when // ANTHROPIC_UPSTREAM_BASE_URL isn't set — letting the live-sandbox e2e // run with no AI credentials. - if (opts.agentType === "claude-agent-sdk") { + const openCodeUsesAnthropic = + opts.agentType === "opencode" && + (opts.aiProvider ?? providerFromModel(opts.model) ?? "anthropic") === "anthropic" && + Boolean(sandboxEnv["ANTHROPIC_UPSTREAM_BASE_URL"]); + if (opts.agentType === "claude-agent-sdk" || openCodeUsesAnthropic) { await startRequestProxy(sandbox, opts.mode, opts.onLog); } @@ -552,6 +600,39 @@ exit 1 } } +export async function ensureOpenCodeBinary( + sandbox: Sandbox, + mode: DeepsecMode, + onLog: (msg: string) => void, +): Promise { + const binaryPath = OPENCODE_BINARY_BY_MODE[mode]; + const link = await sandbox.runCommand({ + cmd: "ln", + args: ["-sfn", binaryPath, "/usr/local/bin/opencode"], + sudo: true, + }); + if (link.exitCode !== 0) { + const stderr = (await link.stderr()).trim(); + throw new Error( + `OpenCode native binary link failed (exit ${link.exitCode})${stderr ? `: ${stderr}` : ""}`, + ); + } + + const result = await sandbox.runCommand({ + cmd: "opencode", + args: ["--version"], + cwd: DEEPSEC_DIR, + }); + const stdout = (await result.stdout()).trim(); + const stderr = (await result.stderr()).trim(); + for (const line of `${stdout}\n${stderr}`.split("\n")) { + if (line.trim()) onLog(` ${line}`); + } + if (result.exitCode !== 0) { + throw new Error(`OpenCode native binary check failed (exit ${result.exitCode})`); + } +} + // --- Native binary remediation (shared helper) --- async function ensureClaudeNativeBinaries( diff --git a/packages/deepsec/src/sandbox/types.ts b/packages/deepsec/src/sandbox/types.ts index 02a6e057..7a3dc39c 100644 --- a/packages/deepsec/src/sandbox/types.ts +++ b/packages/deepsec/src/sandbox/types.ts @@ -20,12 +20,14 @@ export interface SandboxConfig { concurrency: number; /** Batch size within each sandbox */ batchSize: number; - /** Agent backend type — claude-agent-sdk, codex, or pi */ + /** Agent backend type — claude-agent-sdk, codex, opencode, or pi */ agentType?: string; - /** Pi provider API key env var for custom gateway/provider routing */ + /** Pi/OpenCode provider API key env var for custom gateway/provider routing */ aiApiKeyEnv?: string; - /** Pi provider base URL for custom gateway/provider routing */ + /** Pi/OpenCode provider base URL for custom gateway/provider routing */ aiBaseUrl?: string; + /** Pi/OpenCode provider override for custom gateway/provider routing */ + aiProvider?: string; /** Host-only credential broker descriptor; never persisted in detached run state. */ brokeredModelCredential?: BrokeredModelCredential; /** Model to use */ diff --git a/packages/deepsec/src/setup/coordinator.ts b/packages/deepsec/src/setup/coordinator.ts index 41f73ccb..0e915e15 100644 --- a/packages/deepsec/src/setup/coordinator.ts +++ b/packages/deepsec/src/setup/coordinator.ts @@ -3,6 +3,7 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; import { dataDir, + existsSettled, type FileRecord, getRegistry, loadAllFileRecords, @@ -407,8 +408,10 @@ export async function runSetupWorkflow( const installInput = installFingerprint(workspaceDir); let installResult: Awaited>; if ( + // The installer is a child process that just finished writing this + // tree, so a lagging mount can still report it missing. !isCheckpointCurrent(state, "install", installInput, () => - fs.existsSync("node_modules/deepsec"), + existsSettled("node_modules/deepsec"), ) ) { installResult = await runPhase(state, reporter, "install", installInput, () => diff --git a/packages/deepsec/src/setup/install.ts b/packages/deepsec/src/setup/install.ts index a44cf75c..20f1fa3c 100644 --- a/packages/deepsec/src/setup/install.ts +++ b/packages/deepsec/src/setup/install.ts @@ -1,6 +1,7 @@ import { spawn, spawnSync } from "node:child_process"; import fs from "node:fs"; import path from "node:path"; +import { existsSettled } from "@deepsec/core"; import { createTerminalLineDecoder, packageInstallProgress } from "./output.js"; export type SupportedPackageManager = "pnpm" | "npm"; @@ -42,17 +43,26 @@ export function selectPackageManager( throw new Error("Deepsec requires pnpm or npm, but neither command is available on PATH"); } -export function probeWorkspaceInstall(workspaceDir: string): { +export function probeWorkspaceInstall( + workspaceDir: string, + // Set when the installer child just wrote this tree: a lagging mount can + // report the files it created as missing for another millisecond or two, so + // a miss is re-checked after settling instead of failing the install. Left + // off for the pre-install probe, where an absent node_modules is the normal + // case and waiting on it is pure cost. + options: { settle?: boolean } = {}, +): { ok: boolean; version?: string; reason?: string; } { + const exists = options.settle ? existsSettled : fs.existsSync; const packageFile = path.join(workspaceDir, "node_modules", "deepsec", "package.json"); - if (!fs.existsSync(packageFile)) return { ok: false, reason: "node_modules/deepsec is missing" }; + if (!exists(packageFile)) return { ok: false, reason: "node_modules/deepsec is missing" }; try { const pkg = JSON.parse(fs.readFileSync(packageFile, "utf8")); for (const relative of ["dist/cli.mjs", "dist/config.mjs"]) { - if (!fs.existsSync(path.join(workspaceDir, "node_modules", "deepsec", relative))) { + if (!exists(path.join(workspaceDir, "node_modules", "deepsec", relative))) { return { ok: false, reason: `deepsec/${relative} is missing` }; } } @@ -125,7 +135,7 @@ export async function ensureWorkspaceInstall(params: { `${packageManager} install failed with exit ${result.status ?? "unknown"}; see the setup log for package-manager output`, ); } - const after = probeWorkspaceInstall(workspaceDir); + const after = probeWorkspaceInstall(workspaceDir, { settle: true }); if (!after.ok) throw new Error(`Install completed but workspace is unusable: ${after.reason}`); params.onProgress?.({ message: `Installed deepsec ${after.version}` }); return { packageManager, version: after.version!, installed: true }; diff --git a/packages/processor/package.json b/packages/processor/package.json index 5f59bb78..8e92f1ae 100644 --- a/packages/processor/package.json +++ b/packages/processor/package.json @@ -15,6 +15,8 @@ "@earendil-works/pi-coding-agent": "0.81.1", "@openai/codex": "^0.156.1", "@openai/codex-sdk": "^0.156.1", - "jsonrepair": "^3.14.0" + "@opencode-ai/sdk": "^1.18.35", + "jsonrepair": "^3.14.0", + "undici": "^8.5.0" } } diff --git a/packages/processor/src/__tests__/opencode-sdk.test.ts b/packages/processor/src/__tests__/opencode-sdk.test.ts new file mode 100644 index 00000000..9aab5ede --- /dev/null +++ b/packages/processor/src/__tests__/opencode-sdk.test.ts @@ -0,0 +1,368 @@ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + buildOpenCodeConfig, + buildOpenCodeV2Config, + formatOpenCodeError, + openCodeServerEnv, + openCodeV2ProviderOverlays, + parseOpenCodeModel, + protocolFromVersionOutput, + resolveOpenCodeAssistantText, + resolveOpenCodeVariant, + shouldUseOpenCodeTextFormat, +} from "../agents/opencode-sdk.js"; + +describe("OpenCodeAgentPlugin configuration", () => { + const savedEnv = { + ANTHROPIC_AUTH_TOKEN: process.env.ANTHROPIC_AUTH_TOKEN, + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY, + ANTHROPIC_BASE_URL: process.env.ANTHROPIC_BASE_URL, + OPENAI_API_KEY: process.env.OPENAI_API_KEY, + OPENAI_BASE_URL: process.env.OPENAI_BASE_URL, + }; + + afterEach(() => { + for (const [key, value] of Object.entries(savedEnv)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + }); + + it("parses provider/model while preserving slashes in the model id", () => { + expect(parseOpenCodeModel("openai/acme/gpt-sec")).toEqual({ + providerID: "openai", + modelID: "acme/gpt-sec", + }); + }); + + it("rejects a bare model name with an actionable example", () => { + expect(() => parseOpenCodeModel("claude-opus")).toThrow(/provider\/model/); + expect(() => parseOpenCodeModel("claude-opus")).toThrow(/anthropic\/claude-opus-4-8/); + }); + + it("maps the shared thinking dial to provider variants", () => { + expect(resolveOpenCodeVariant("anthropic", "xhigh")).toBe("max"); + expect(resolveOpenCodeVariant("anthropic", "medium")).toBe("high"); + expect(resolveOpenCodeVariant("openai", "low")).toBe("low"); + expect(resolveOpenCodeVariant("google", "minimal")).toBe("low"); + expect(resolveOpenCodeVariant("google", "high")).toBe("high"); + expect(resolveOpenCodeVariant("custom")).toBeUndefined(); + expect(resolveOpenCodeVariant("custom", "medium")).toBe("medium"); + }); + + it("uses text output when Anthropic thinking conflicts with forced structured output", () => { + expect( + shouldUseOpenCodeTextFormat("anthropic", "high", { + type: "json_schema", + schema: { type: "object" }, + }), + ).toBe(true); + expect( + shouldUseOpenCodeTextFormat("anthropic", undefined, { + type: "json_schema", + schema: { type: "object" }, + }), + ).toBe(false); + expect( + shouldUseOpenCodeTextFormat("openai", "high", { + type: "json_schema", + schema: { type: "object" }, + }), + ).toBe(false); + expect(shouldUseOpenCodeTextFormat("anthropic", "high", { type: "text" })).toBe(false); + }); + + it("preserves nested network error details", () => { + const cause = Object.assign(new Error("Headers Timeout Error"), { + code: "UND_ERR_HEADERS_TIMEOUT", + }); + const error = new TypeError("fetch failed", { cause }); + + expect(formatOpenCodeError(error)).toBe( + "fetch failed: Headers Timeout Error (UND_ERR_HEADERS_TIMEOUT)", + ); + }); + + it("uses plain text when a provider skips OpenCode's StructuredOutput tool", () => { + expect( + resolveOpenCodeAssistantText( + { + error: { + name: "StructuredOutputError", + data: { + message: "Model did not produce structured output", + retries: 0, + }, + }, + }, + '[{"filePath":"src/a.ts","findings":[]}]', + ), + ).toEqual({ + resultText: '[{"filePath":"src/a.ts","findings":[]}]', + recoveredStructuredText: true, + }); + }); + + it("keeps a structured-output failure when the provider returned no text", () => { + expect(() => + resolveOpenCodeAssistantText( + { + error: { + name: "StructuredOutputError", + data: { + message: "Model did not produce structured output", + retries: 0, + }, + }, + }, + "", + ), + ).toThrow(/StructuredOutputError: Model did not produce structured output/); + }); + + it("enforces read-only tools, capped steps, and structured provider routing", () => { + process.env.ANTHROPIC_AUTH_TOKEN = "do-not-inline-this-secret"; + process.env.ANTHROPIC_BASE_URL = "https://ai-gateway.example"; + + const config = buildOpenCodeConfig({ + model: "anthropic/claude-opus-4-8", + maxTurns: 42, + thinkingLevel: "xhigh", + }); + const main = config.agent?.deepsec; + + expect(config.default_agent).toBe("deepsec"); + expect(config.share).toBe("disabled"); + expect(config.autoupdate).toBe(false); + expect(config.tools).toMatchObject({ + read: true, + glob: true, + grep: true, + list: true, + bash: false, + edit: false, + task: false, + webfetch: false, + }); + expect(config.permission).toMatchObject({ + "*": "deny", + read: "allow", + bash: "deny", + edit: "deny", + external_directory: "deny", + }); + expect(main?.steps).toBe(42); + expect(main?.variant).toBe("max"); + expect(main?.permission).toEqual(config.permission); + expect(config.provider?.anthropic?.options).toMatchObject({ + apiKey: "{env:ANTHROPIC_AUTH_TOKEN}", + baseURL: "https://ai-gateway.example/v1", + }); + expect(JSON.stringify(config)).not.toContain("do-not-inline-this-secret"); + }); + + it("does not duplicate the Anthropic API version path", () => { + process.env.ANTHROPIC_BASE_URL = "https://api.anthropic.com/v1/"; + + const config = buildOpenCodeConfig({ + model: "anthropic/claude-haiku-4-5", + }); + + expect(config.provider?.anthropic?.options?.baseURL).toBe("https://api.anthropic.com/v1"); + }); + + it("supports a custom provider override without changing the model id", () => { + process.env.MARTIAN_API_KEY = "secret"; + const config = buildOpenCodeConfig({ + model: "openai/org/model", + aiProvider: "martian", + aiBaseUrl: "https://martian.example/v1", + aiApiKeyEnv: "MARTIAN_API_KEY", + aiHeaders: { "x-team": "security" }, + }); + + expect(config.model).toBe("martian/org/model"); + expect(config.provider?.martian).toEqual({ + options: { + apiKey: "{env:MARTIAN_API_KEY}", + baseURL: "https://martian.example/v1", + }, + headers: { "x-team": "security" }, + }); + }); +}); + +describe("OpenCode v2 support", () => { + const savedEnv = { + MARTIAN_API_KEY: process.env.MARTIAN_API_KEY, + ANTHROPIC_AUTH_TOKEN: process.env.ANTHROPIC_AUTH_TOKEN, + ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY, + ANTHROPIC_BASE_URL: process.env.ANTHROPIC_BASE_URL, + OPENAI_BASE_URL: process.env.OPENAI_BASE_URL, + }; + + beforeEach(() => { + delete process.env.ANTHROPIC_AUTH_TOKEN; + delete process.env.ANTHROPIC_API_KEY; + delete process.env.ANTHROPIC_BASE_URL; + delete process.env.OPENAI_BASE_URL; + }); + + afterEach(() => { + for (const [key, value] of Object.entries(savedEnv)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + }); + + it("maps opencode --version output onto the runtime protocol", () => { + expect(protocolFromVersionOutput("opencode v2.0.24")).toBe("v2"); + expect(protocolFromVersionOutput("opencode v2.1.0-alpha.3")).toBe("v2"); + expect(protocolFromVersionOutput("1.18.35")).toBe("v1"); + expect(protocolFromVersionOutput("opencode 1.18.35")).toBe("v1"); + expect(protocolFromVersionOutput("")).toBe("v1"); + expect(protocolFromVersionOutput("not-a-version")).toBe("v1"); + }); + + it("builds a read-only v2 config with ordered permission rules", () => { + const config = buildOpenCodeV2Config({ + model: "anthropic/claude-opus-4-8", + maxTurns: 42, + }); + + expect(config.share).toBe("disabled"); + expect(config.update).toBe("disable"); + expect(config.snapshots).toBe(false); + expect(config.model).toBe("anthropic/claude-opus-4-8"); + expect(config.default_agent).toBe("deepsec"); + + const readOnly = [ + { action: "*", resource: "*", effect: "deny" }, + { action: "read", resource: "*", effect: "allow" }, + { action: "glob", resource: "*", effect: "allow" }, + { action: "grep", resource: "*", effect: "allow" }, + { action: "list", resource: "*", effect: "allow" }, + ]; + expect(config.permissions).toEqual(readOnly); + + const main = (config.agents as Record)["deepsec"]; + expect(main.mode).toBe("primary"); + expect(main.steps).toBe(42); + expect(main.permissions).toEqual(readOnly); + expect(String(main.system)).toContain("static source inspection only"); + + const json = (config.agents as Record)["deepsec-json"]; + expect(json.hidden).toBe(true); + expect(json.steps).toBe(1); + expect(json.permissions).toEqual([{ action: "*", resource: "*", effect: "deny" }]); + }); + + it("remaps a custom provider override onto the standard provider env for v2", () => { + process.env.MARTIAN_API_KEY = "martian-secret"; + const env = openCodeServerEnv( + { + model: "openai/gpt-5.5", + aiBaseUrl: "https://api.withmartian.com/v1", + aiApiKeyEnv: "MARTIAN_API_KEY", + }, + "v2", + ); + expect(env).toEqual({ + OPENAI_API_KEY: "martian-secret", + OPENAI_BASE_URL: "https://api.withmartian.com/v1", + }); + + const anthropic = openCodeServerEnv( + { + model: "anthropic/claude-opus-4-8", + aiProvider: "anthropic", + aiBaseUrl: "https://proxy.example", + aiApiKeyEnv: "MARTIAN_API_KEY", + }, + "v2", + ); + expect(anthropic).toEqual({ + ANTHROPIC_API_KEY: "martian-secret", + ANTHROPIC_BASE_URL: "https://proxy.example", + }); + }); + + it("rejects custom provider headers on the v2 runtime", () => { + process.env.MARTIAN_API_KEY = "martian-secret"; + expect(() => + openCodeServerEnv( + { + model: "openai/gpt-5.5", + aiBaseUrl: "https://api.withmartian.com/v1", + aiApiKeyEnv: "MARTIAN_API_KEY", + aiHeaders: { "x-team": "security" }, + }, + "v2", + ), + ).toThrow(/--ai-header/); + }); + + it("passes no extra server env for v1 or uncustomized routes", () => { + expect(openCodeServerEnv({ model: "anthropic/claude-opus-4-8" }, "v1")).toEqual({}); + expect(openCodeServerEnv({ model: "anthropic/claude-opus-4-8" }, "v2")).toEqual({}); + expect( + openCodeServerEnv({ model: "openai/gpt-5.5", aiBaseUrl: "https://x.example" }, "v2"), + ).toEqual({}); + }); + + it("bridges the gateway bearer token to the v2 anthropic env var", () => { + process.env.ANTHROPIC_AUTH_TOKEN = "vck_gateway_token"; + process.env.ANTHROPIC_BASE_URL = "https://ai-gateway.vercel.sh"; + const env = openCodeServerEnv({ model: "anthropic/claude-opus-4-8" }, "v2"); + expect(env).toEqual({ ANTHROPIC_API_KEY: "vck_gateway_token" }); + + // An explicit API key wins — no bridge needed. + process.env.ANTHROPIC_API_KEY = "sk-ant-direct"; + expect(openCodeServerEnv({ model: "anthropic/claude-opus-4-8" }, "v2")).toEqual({}); + }); + + it("translates the gateway expansion into v2 provider baseURL overlays", () => { + process.env.ANTHROPIC_BASE_URL = "https://ai-gateway.vercel.sh"; + process.env.OPENAI_BASE_URL = "https://ai-gateway.vercel.sh/v1"; + const overlays = openCodeV2ProviderOverlays({ model: "anthropic/claude-opus-4-8" }); + expect(overlays).toEqual({ + anthropic: { settings: { baseURL: "https://ai-gateway.vercel.sh" } }, + openai: { settings: { baseURL: "https://ai-gateway.vercel.sh/v1" } }, + }); + expect(openCodeServerEnv({ model: "anthropic/claude-opus-4-8" }, "v2")).toEqual({}); + }); + + it("includes provider overlays in the v2 config content", () => { + process.env.ANTHROPIC_BASE_URL = "https://ai-gateway.vercel.sh"; + const config = buildOpenCodeV2Config({ model: "anthropic/claude-opus-4-8" }); + expect(config.providers).toEqual({ + anthropic: { settings: { baseURL: "https://ai-gateway.vercel.sh" } }, + }); + }); + + it("redirects the model's own provider for custom v2 routes", () => { + const overlays = openCodeV2ProviderOverlays({ + model: "openai/gpt-5.5", + aiProvider: "martian", + aiBaseUrl: "https://api.withmartian.com/v1", + aiApiKeyEnv: "MARTIAN_API_KEY", + }); + expect(overlays).toEqual({ + openai: { settings: { baseURL: "https://api.withmartian.com/v1" } }, + }); + }); + + it("requires the configured credential env for custom v2 routes", () => { + delete process.env.MARTIAN_API_KEY; + expect(() => + openCodeServerEnv( + { + model: "openai/gpt-5.5", + aiBaseUrl: "https://api.withmartian.com/v1", + aiApiKeyEnv: "MARTIAN_API_KEY", + }, + "v2", + ), + ).toThrow(/MARTIAN_API_KEY/); + }); +}); diff --git a/packages/processor/src/__tests__/registry.test.ts b/packages/processor/src/__tests__/registry.test.ts index 2f9e06c1..32607d7b 100644 --- a/packages/processor/src/__tests__/registry.test.ts +++ b/packages/processor/src/__tests__/registry.test.ts @@ -56,7 +56,8 @@ describe("createDefaultAgentRegistry", () => { const registry = createDefaultAgentRegistry(); expect(registry.get("claude-agent-sdk")).toBeDefined(); expect(registry.get("codex")).toBeDefined(); + expect(registry.get("opencode")).toBeDefined(); expect(registry.get("pi")).toBeDefined(); - expect(registry.types().sort()).toEqual(["claude-agent-sdk", "codex", "pi"]); + expect(registry.types().sort()).toEqual(["claude-agent-sdk", "codex", "opencode", "pi"]); }); }); diff --git a/packages/processor/src/__tests__/shared.test.ts b/packages/processor/src/__tests__/shared.test.ts index 8a1ba868..92208441 100644 --- a/packages/processor/src/__tests__/shared.test.ts +++ b/packages/processor/src/__tests__/shared.test.ts @@ -3,10 +3,12 @@ import os from "node:os"; import path from "node:path"; import { afterEach, beforeEach, describe, expect, it } from "vitest"; import { + AgentPolicyRefusalError, attributionHeaders, buildInvestigateFieldRepairPrompt, buildInvestigateJsonRepairPrompt, buildRevalidateJsonRepairPrompt, + classifyPolicyRefusal, classifyQuotaError, formatJsonRepairFailureDebugText, isGatewayBaseUrl, @@ -201,6 +203,50 @@ describe("QuotaExhaustedError", () => { }); }); +describe("classifyPolicyRefusal", () => { + it("recognizes the Claude Code usage-policy refusal (issue #92)", () => { + const text = + "API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup). Try rephrasing the request or attempting a different approach. Request ID: req_123"; + expect(classifyPolicyRefusal(text)).toBe("anthropic-usage-policy"); + }); + + it("recognizes OpenAI safety-system rejections", () => { + expect(classifyPolicyRefusal('{"error":{"code":"invalid_prompt"}}')).toBe( + "openai-safety-system", + ); + expect(classifyPolicyRefusal("The prompt was rejected as a result of our safety system.")).toBe( + "openai-safety-system", + ); + }); + + it("recognizes generic content-policy envelopes", () => { + expect(classifyPolicyRefusal('{"error":{"type":"content_policy_violation"}}')).toBe( + "content-policy", + ); + }); + + it("returns undefined for ordinary malformed output and prose", () => { + // Strict on purpose: relabeling broken JSON as a refusal would report + // zero coverage where the real problem is formatting — worse than the + // status quo. Misses degrade to the existing fail-loud parse error. + expect(classifyPolicyRefusal("not JSON at all")).toBeUndefined(); + expect(classifyPolicyRefusal('```json\n[{"filePath":')).toBeUndefined(); + expect(classifyPolicyRefusal("")).toBeUndefined(); + expect(classifyPolicyRefusal("You've hit your usage limit.")).toBeUndefined(); + }); +}); + +describe("AgentPolicyRefusalError", () => { + it("carries source + raw, and the message includes both", () => { + const e = new AgentPolicyRefusalError("anthropic-usage-policy", "API Error: Claude Code…"); + expect(e.source).toBe("anthropic-usage-policy"); + expect(e.rawMessage).toContain("API Error"); + expect(e.message).toContain("anthropic-usage-policy"); + expect(e.name).toBe("AgentPolicyRefusalError"); + expect(e instanceof Error).toBe(true); + }); +}); + describe("isUsingAiGateway", () => { // We mutate process.env directly — capture and restore so tests don't // bleed environment into each other. @@ -403,6 +449,28 @@ describe("parseInvestigateResults", () => { ); }); + it("throws AgentPolicyRefusalError (not a parse error) on a provider policy refusal", () => { + // A refusal must never reach jsonrepair or surface as a formatting + // problem: the batch has zero coverage, and the repair follow-up would + // accept an empty findings array as a valid answer. + const refusal = + "API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup). Try rephrasing the request or attempting a different approach."; + expect(() => parseInvestigateResults(refusal, batch)).toThrow(AgentPolicyRefusalError); + }); + + it("does NOT flag valid findings JSON that quotes refusal text as evidence", () => { + // Classification runs only inside the JSON.parse catch, so a finding + // citing a refusal string parses like any other. + const finding = validFinding({ + description: + "Endpoint relays the upstream string 'unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup)' verbatim to clients.", + }); + const text = `\`\`\`json\n[{"filePath":"a.ts","findings":[${JSON.stringify(finding)}]}]\n\`\`\``; + const out = parseInvestigateResults(text, batch); + expect(out.results.find((r) => r.filePath === "a.ts")?.findings).toHaveLength(1); + expect(out.invalid).toEqual([]); + }); + it("salvages valid findings and reports invalid ones instead of dropping the file", () => { // One bad enum used to poison the whole file record downstream: the // read-side schema rejected the record and every valid finding in it @@ -579,6 +647,12 @@ describe("parseRevalidateVerdicts", () => { it("throws on parse failure", () => { expect(() => parseRevalidateVerdicts("garbage")).toThrow(/wasn't parseable JSON/); }); + + it("throws AgentPolicyRefusalError on a provider policy refusal", () => { + const refusal = + "API Error: Claude Code is unable to respond to this request, which appears to violate our Usage Policy (https://www.anthropic.com/legal/aup)."; + expect(() => parseRevalidateVerdicts(refusal)).toThrow(AgentPolicyRefusalError); + }); }); describe("writeParseFailureDebug", () => { diff --git a/packages/processor/src/__tests__/triage-parse.test.ts b/packages/processor/src/__tests__/triage-parse.test.ts new file mode 100644 index 00000000..06ca6606 --- /dev/null +++ b/packages/processor/src/__tests__/triage-parse.test.ts @@ -0,0 +1,124 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import type { FileRecord } from "@deepsec/core"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +// triage() drives the Claude Agent SDK's `query` directly, so stub the SDK to +// yield a canned (here: malformed) result payload. +const sdk = vi.hoisted(() => ({ resultText: "" })); +vi.mock("@anthropic-ai/claude-agent-sdk", () => ({ + query: () => + (async function* () { + yield { type: "result", subtype: "success", result: sdk.resultText }; + })(), +})); + +import { triage } from "../triage.js"; + +function setupProject(): { + projectId: string; + writeRecord: (rec: FileRecord) => void; + readRecord: (rel: string) => FileRecord; +} { + const projectId = "triage-parse-test"; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-triage-parse-")); + const targetRoot = path.join(tmp, "target"); + const dataRoot = path.join(tmp, "data"); + fs.mkdirSync(targetRoot, { recursive: true }); + fs.mkdirSync(path.join(dataRoot, projectId, "files"), { recursive: true }); + fs.writeFileSync( + path.join(dataRoot, projectId, "project.json"), + JSON.stringify({ projectId, rootPath: targetRoot, createdAt: new Date().toISOString() }), + ); + process.env.DEEPSEC_DATA_ROOT = dataRoot; + + const recordPath = (rel: string) => path.join(dataRoot, projectId, "files", `${rel}.json`); + const writeRecord = (rec: FileRecord): void => { + const p = recordPath(rec.filePath); + fs.mkdirSync(path.dirname(p), { recursive: true }); + fs.writeFileSync(p, JSON.stringify(rec)); + }; + const readRecord = (rel: string): FileRecord => + JSON.parse(fs.readFileSync(recordPath(rel), "utf-8")); + return { projectId, writeRecord, readRecord }; +} + +function recordWithFinding(projectId: string, filePath: string, title: string): FileRecord { + return { + filePath, + projectId, + candidates: [{ vulnSlug: "ssrf", lineNumbers: [1], snippet: "// stub", matchedPattern: "p" }], + lastScannedAt: new Date().toISOString(), + lastScannedRunId: "scan-fixture", + fileHash: "fixture-hash", + findings: [ + { + severity: "MEDIUM", + vulnSlug: "ssrf", + title, + description: "x", + lineNumbers: [1], + recommendation: "x", + confidence: "high", + }, + ], + analysisHistory: [], + status: "analyzed", + }; +} + +describe("triage JSON-parse robustness", () => { + let prevDataRoot: string | undefined; + + beforeEach(() => { + prevDataRoot = process.env.DEEPSEC_DATA_ROOT; + }); + + afterEach(() => { + if (prevDataRoot === undefined) delete process.env.DEEPSEC_DATA_ROOT; + else process.env.DEEPSEC_DATA_ROOT = prevDataRoot; + }); + + it("recovers a trailing-comma verdict array via the tolerant parser", async () => { + const fx = setupProject(); + fx.writeRecord(recordWithFinding(fx.projectId, "a.ts", "SSRF in fetch")); + + // Trailing comma before the closing brace — invalid strict JSON, but a + // jsonrepair-class error the tolerant parser recovers. Before the fix a + // bare JSON.parse threw, was swallowed, and the finding was left untriaged. + sdk.resultText = + '```json\n[{"title":"SSRF in fetch","priority":"P0",' + + '"exploitability":"trivial","impact":"high","reasoning":"r",}]\n```'; + + const result = await triage({ projectId: fx.projectId, concurrency: 1 }); + + expect(fx.readRecord("a.ts").findings[0].triage?.priority).toBe("P0"); + expect(result.triaged).toBe(1); + }); + + it("surfaces a batch as failed when the output is unparseable, not silently empty", async () => { + const fx = setupProject(); + fx.writeRecord(recordWithFinding(fx.projectId, "a.ts", "SSRF in fetch")); + + // Prose the model returned instead of JSON. strict JSON.parse rejects it + // (before the fix, that was swallowed and printed as "0 triaged"), and + // jsonrepair can only coerce it to a bare string — not an array — so the + // tolerant parser fails loud. + sdk.resultText = "I cannot comply with this request"; + + const messages: string[] = []; + const result = await triage({ + projectId: fx.projectId, + concurrency: 1, + onProgress: (p) => messages.push(p.message), + }); + + // The batch is reported as failed (before the fix it printed "0 triaged" + // as a success), and the finding stays untriaged. + expect(messages.some((m) => /failed/i.test(m))).toBe(true); + expect(messages.some((m) => /\b0 triaged\b/.test(m))).toBe(false); + expect(fx.readRecord("a.ts").findings[0].triage).toBeUndefined(); + expect(result.triaged).toBe(0); + }); +}); diff --git a/packages/processor/src/__tests__/triage.test.ts b/packages/processor/src/__tests__/triage.test.ts new file mode 100644 index 00000000..486fd704 --- /dev/null +++ b/packages/processor/src/__tests__/triage.test.ts @@ -0,0 +1,133 @@ +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import type { FileRecord } from "@deepsec/core"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +// triage() drives the Claude Agent SDK's `query` directly (not via the plugin +// agent system), so stub the SDK to yield a canned verdict payload. +const sdk = vi.hoisted(() => ({ resultText: "" })); +vi.mock("@anthropic-ai/claude-agent-sdk", () => ({ + query: () => + (async function* () { + yield { type: "result", subtype: "success", result: sdk.resultText }; + })(), +})); + +import { triage } from "../triage.js"; + +function setupProject(): { + projectId: string; + writeRecord: (rec: FileRecord) => void; + readRecord: (rel: string) => FileRecord; +} { + const projectId = "triage-test"; + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "deepsec-triage-")); + const targetRoot = path.join(tmp, "target"); + const dataRoot = path.join(tmp, "data"); + fs.mkdirSync(targetRoot, { recursive: true }); + fs.mkdirSync(path.join(dataRoot, projectId, "files"), { recursive: true }); + fs.writeFileSync( + path.join(dataRoot, projectId, "project.json"), + JSON.stringify({ projectId, rootPath: targetRoot, createdAt: new Date().toISOString() }), + ); + process.env.DEEPSEC_DATA_ROOT = dataRoot; + + const recordPath = (rel: string) => path.join(dataRoot, projectId, "files", `${rel}.json`); + const writeRecord = (rec: FileRecord): void => { + const p = recordPath(rec.filePath); + fs.mkdirSync(path.dirname(p), { recursive: true }); + fs.writeFileSync(p, JSON.stringify(rec)); + }; + const readRecord = (rel: string): FileRecord => + JSON.parse(fs.readFileSync(recordPath(rel), "utf-8")); + return { projectId, writeRecord, readRecord }; +} + +function recordWithFinding(projectId: string, filePath: string, title: string): FileRecord { + return { + filePath, + projectId, + candidates: [ + { vulnSlug: "missing-authz", lineNumbers: [1], snippet: "// stub", matchedPattern: "p" }, + ], + lastScannedAt: new Date().toISOString(), + lastScannedRunId: "scan-fixture", + fileHash: "fixture-hash", + findings: [ + { + severity: "MEDIUM", + vulnSlug: "missing-authz", + title, + description: "x", + lineNumbers: [1], + recommendation: "x", + confidence: "high", + }, + ], + analysisHistory: [], + status: "analyzed", + }; +} + +describe("triage verdict correlation", () => { + let prevDataRoot: string | undefined; + + beforeEach(() => { + prevDataRoot = process.env.DEEPSEC_DATA_ROOT; + }); + + afterEach(() => { + if (prevDataRoot === undefined) delete process.env.DEEPSEC_DATA_ROOT; + else process.env.DEEPSEC_DATA_ROOT = prevDataRoot; + }); + + it("maps each verdict to its own finding by id when titles collide", async () => { + const fx = setupProject(); + fx.writeRecord(recordWithFinding(fx.projectId, "a.ts", "Missing authorization check")); + fx.writeRecord(recordWithFinding(fx.projectId, "b.ts", "Missing authorization check")); + + sdk.resultText = [ + "```json", + JSON.stringify([ + { + id: 1, + title: "Missing authorization check", + priority: "P0", + exploitability: "trivial", + impact: "high", + reasoning: "one", + }, + { + id: 2, + title: "Missing authorization check", + priority: "skip", + exploitability: "difficult", + impact: "low", + reasoning: "two", + }, + ]), + "```", + ].join("\n"); + + const result = await triage({ projectId: fx.projectId, concurrency: 1 }); + + const a = fx.readRecord("a.ts").findings[0]; + const b = fx.readRecord("b.ts").findings[0]; + + // Both same-titled findings must be triaged. Before the fix, title-matching + // collapsed both verdicts onto the first finding, leaving the other one + // untouched. + expect(a.triage).toBeDefined(); + expect(b.triage).toBeDefined(); + + // And each keeps its own distinct verdict (P0 vs skip), rather than one + // verdict overwriting the other. + const priorities = [a.triage?.priority, b.triage?.priority].sort(); + expect(priorities).toEqual(["P0", "skip"]); + + expect(result.triaged).toBe(2); + expect(result.p0).toBe(1); + expect(result.skip).toBe(1); + }); +}); diff --git a/packages/processor/src/agents/claude-agent-sdk.ts b/packages/processor/src/agents/claude-agent-sdk.ts index 0539444a..93dc6a3b 100644 --- a/packages/processor/src/agents/claude-agent-sdk.ts +++ b/packages/processor/src/agents/claude-agent-sdk.ts @@ -1,6 +1,7 @@ import { query, type SandboxSettings } from "@anthropic-ai/claude-agent-sdk"; import type { RefusalReport } from "@deepsec/core"; import { + AgentPolicyRefusalError, attributionHeaders, backoff, buildInvestigateJsonRepairPrompt, @@ -493,6 +494,17 @@ export class ClaudeAgentSdkPlugin implements AgentPlugin { try { parsed = parseInvestigateResults(resultText, batch); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } yield { type: "thinking" as const, message: "Claude returned non-JSON investigation output; requesting JSON-only repair", @@ -727,6 +739,17 @@ export class ClaudeAgentSdkPlugin implements AgentPlugin { try { verdicts = parseRevalidateVerdicts(resultText); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } yield { type: "thinking" as const, message: "Claude returned non-JSON revalidation output; requesting JSON-only repair", diff --git a/packages/processor/src/agents/codex-sdk.ts b/packages/processor/src/agents/codex-sdk.ts index 9d3a778c..39d58aeb 100644 --- a/packages/processor/src/agents/codex-sdk.ts +++ b/packages/processor/src/agents/codex-sdk.ts @@ -13,6 +13,7 @@ import { type ThreadItem, } from "@openai/codex-sdk"; import { + AgentPolicyRefusalError, attributionHeaders, backoff, buildInvestigateJsonRepairPrompt, @@ -319,8 +320,16 @@ function buildCodexInvocation(): CodexInvocation { // codex use its default openai provider against their session). Sandbox // workers always go gateway; the preflight ensures a token is present // before we ever get here in that path. - const haveApiToken = !!(process.env.OPENAI_API_KEY || process.env.ANTHROPIC_AUTH_TOKEN); - const subscriptionHome = haveApiToken ? null : findCodexSubscriptionAuth(); + // + // Precedence matches the codex CLI itself: a ChatGPT/Codex subscription + // login wins over an ambient OPENAI_API_KEY / ANTHROPIC_AUTH_TOKEN, so a + // stray API key in the environment no longer silently burns API credits + // while a valid subscription sits unused (#32). An explicit base URL + // (OPENAI_BASE_URL / ANTHROPIC_BASE_URL — e.g. the AI Gateway expansion + // or a direct-provider route) means the token was routed deliberately, + // so in that case the token wins and the subscription is ignored. + const hasExplicitRouting = Boolean(process.env.OPENAI_BASE_URL ?? process.env.ANTHROPIC_BASE_URL); + const subscriptionHome = hasExplicitRouting ? null : findCodexSubscriptionAuth(); const codexHome = makeCodexHome(); if (subscriptionHome) { @@ -955,6 +964,17 @@ export class CodexAgentSdkPlugin implements AgentPlugin { try { parsedOutcome = parseInvestigateResults(resultText, batch); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } yield { type: "thinking" as const, message: "Codex returned non-JSON investigation output; requesting JSON-only repair", @@ -1240,6 +1260,17 @@ export class CodexAgentSdkPlugin implements AgentPlugin { try { verdicts = parseRevalidateVerdicts(resultText); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } yield { type: "thinking" as const, message: "Codex returned non-JSON revalidation output; requesting JSON-only repair", diff --git a/packages/processor/src/agents/opencode-sdk.ts b/packages/processor/src/agents/opencode-sdk.ts new file mode 100644 index 00000000..7e594599 --- /dev/null +++ b/packages/processor/src/agents/opencode-sdk.ts @@ -0,0 +1,1706 @@ +import { spawn } from "node:child_process"; +import crypto from "node:crypto"; +import net from "node:net"; +import type { RefusalReport } from "@deepsec/core"; +import { + type AssistantMessage, + type Config, + createOpencodeClient, + type OpencodeClient, + type OutputFormat, + type Part, + type PermissionConfig, +} from "@opencode-ai/sdk/v2"; +import { Agent as UndiciAgent, fetch as undiciFetch } from "undici"; +import { + AgentPolicyRefusalError, + backoff, + buildInvestigateJsonRepairPrompt, + buildInvestigatePrompt, + buildRevalidateJsonRepairPrompt, + buildRevalidatePrompt, + classifyQuotaError, + formatJsonRepairFailureDebugText, + isTransientError, + jsonRepairFailureError, + MAX_ATTEMPTS, + type ParsedInvestigateResults, + parseInvestigateResults, + parseRefusalReport, + parseRevalidateVerdicts, + QuotaExhaustedError, + REFUSAL_FOLLOWUP_PROMPT, + runInvestigateFieldRepairLoop, + writeParseFailureDebug, +} from "./shared.js"; +import type { + AgentPlugin, + AgentProgress, + BatchMeta, + InvestigateOutput, + InvestigateParams, + InvestigateResult, + RevalidateOutput, + RevalidateParams, + RevalidateVerdict, +} from "./types.js"; + +const DEFAULT_MODEL = "anthropic/claude-opus-4-8"; +const DEFAULT_THINKING_LEVEL = "xhigh"; +const DEFAULT_MAX_TURNS = 150; +const MAIN_AGENT = "deepsec"; +const JSON_AGENT = "deepsec-json"; + +const DEEPSEC_SYSTEM_NOTE = + "You are running inside the OpenCode harness for deepsec. Perform static source inspection only. Do not run the target application, execute shell commands, send network requests, modify files, or attempt exploitation. Return only the requested JSON value."; + +interface OpenCodeAgentConfig { + model?: string; + maxTurns?: number; + thinkingLevel?: string; + aiProvider?: string; + aiBaseUrl?: string; + aiApiKeyEnv?: string; + aiHeaders?: Record; +} + +interface OpenCodeModelRef { + providerID: string; + modelID: string; +} + +/** + * OpenCode runtime generation. The v1 runtime (npm `opencode-ai` 1.x, also + * bundled for sandbox workers) serves the legacy HTTP API (`POST /session`, + * `POST /session/:id/message`, structured output formats). OpenCode v2 + * (2.0.x, e.g. the `anomalyco/tap/opencode-v2` Homebrew package) serves a + * new Effect HttpApi under `/api/…` with mandatory basic auth, an async + * prompt/wait inbox, and no structured-output format — results arrive as + * plain text that the shared JSON-repair pipeline validates. + */ +type OpenCodeProtocol = "v1" | "v2"; + +interface OpenCodeRunContext { + protocol: OpenCodeProtocol; + client: OpencodeClient | undefined; + server: { url: string; close(): void }; + dispatcher: UndiciAgent; + authHeader: string; + sessionID: string; + directory: string; + model: { providerID: string; modelID: string; variant?: string }; + controller: AbortController; + detachParentAbort: () => void; +} + +interface OpenCodePromptResult { + resultText: string; + meta: Partial; + turnCount: number; + toolUseCount: number; + progress: AgentProgress[]; +} + +interface OpenCodeResolvedText { + resultText: string; + recoveredStructuredText: boolean; +} + +const INVESTIGATE_FORMAT: OutputFormat = { + type: "json_schema", + retryCount: 2, + schema: { + type: "array", + items: { + type: "object", + additionalProperties: false, + required: ["filePath", "findings"], + properties: { + filePath: { type: "string" }, + findings: { + type: "array", + items: { + type: "object", + additionalProperties: false, + required: [ + "severity", + "vulnSlug", + "title", + "description", + "lineNumbers", + "recommendation", + "confidence", + ], + properties: { + severity: { + type: "string", + enum: ["CRITICAL", "HIGH", "MEDIUM", "HIGH_BUG", "BUG"], + }, + vulnSlug: { type: "string" }, + title: { type: "string" }, + description: { type: "string" }, + lineNumbers: { + type: "array", + items: { type: "integer" }, + }, + recommendation: { type: "string" }, + confidence: { + type: "string", + enum: ["high", "medium", "low"], + }, + }, + }, + }, + }, + }, + }, +}; + +const REVALIDATE_FORMAT: OutputFormat = { + type: "json_schema", + retryCount: 2, + schema: { + type: "array", + items: { + type: "object", + additionalProperties: false, + required: ["filePath", "title", "verdict", "reasoning"], + properties: { + filePath: { type: "string" }, + title: { type: "string" }, + verdict: { + type: "string", + enum: ["true-positive", "false-positive", "fixed", "uncertain", "duplicate"], + }, + reasoning: { type: "string" }, + adjustedSeverity: { + type: "string", + enum: ["CRITICAL", "HIGH", "MEDIUM", "HIGH_BUG", "BUG"], + }, + duplicateOf: { type: "string" }, + }, + }, + }, +}; + +const TEXT_FORMAT: OutputFormat = { type: "text" }; + +const READ_ONLY_PERMISSION: PermissionConfig = { + "*": "deny", + read: "allow", + glob: "allow", + grep: "allow", + list: "allow", + external_directory: "deny", + bash: "deny", + edit: "deny", + task: "deny", + todowrite: "deny", + question: "deny", + webfetch: "deny", + websearch: "deny", + lsp: "deny", + skill: "deny", + doom_loop: "allow", +}; + +const READ_ONLY_TOOLS: Record = { + read: true, + glob: true, + grep: true, + list: true, + bash: false, + edit: false, + write: false, + patch: false, + task: false, + todowrite: false, + question: false, + webfetch: false, + websearch: false, + lsp: false, + skill: false, +}; + +const NO_TOOLS = Object.fromEntries(Object.keys(READ_ONLY_TOOLS).map((name) => [name, false])); + +function readConfig(config: Record): OpenCodeAgentConfig { + return { + model: typeof config.model === "string" ? config.model : undefined, + maxTurns: typeof config.maxTurns === "number" ? config.maxTurns : undefined, + thinkingLevel: typeof config.thinkingLevel === "string" ? config.thinkingLevel : undefined, + aiProvider: typeof config.aiProvider === "string" ? config.aiProvider : undefined, + aiBaseUrl: typeof config.aiBaseUrl === "string" ? config.aiBaseUrl : undefined, + aiApiKeyEnv: typeof config.aiApiKeyEnv === "string" ? config.aiApiKeyEnv : undefined, + aiHeaders: + config.aiHeaders && typeof config.aiHeaders === "object" && !Array.isArray(config.aiHeaders) + ? (config.aiHeaders as Record) + : undefined, + }; +} + +export function parseOpenCodeModel(model: string): OpenCodeModelRef { + const slash = model.indexOf("/"); + if (slash <= 0 || slash === model.length - 1) { + throw new Error( + `OpenCode model must use provider/model format, got "${model}". ` + + `For example: ${DEFAULT_MODEL}.`, + ); + } + return { + providerID: model.slice(0, slash), + modelID: model.slice(slash + 1), + }; +} + +export function resolveOpenCodeVariant( + providerID: string, + thinkingLevel?: string, +): string | undefined { + const explicit = thinkingLevel !== undefined; + const level = thinkingLevel ?? DEFAULT_THINKING_LEVEL; + const provider = providerID.toLowerCase(); + + if (provider === "anthropic") return level === "xhigh" ? "max" : "high"; + if (provider === "openai") return level; + if (provider === "google") { + return level === "minimal" || level === "low" ? "low" : "high"; + } + return explicit ? level : undefined; +} + +export function shouldUseOpenCodeTextFormat( + providerID: string, + variant: string | undefined, + format: OutputFormat, +): boolean { + return ( + providerID.toLowerCase() === "anthropic" && Boolean(variant) && format.type === "json_schema" + ); +} + +function providerCredentialEnv(providerID: string, cfg: OpenCodeAgentConfig): string | undefined { + if (cfg.aiApiKeyEnv) return cfg.aiApiKeyEnv; + if (providerID === "anthropic") { + if (process.env.ANTHROPIC_AUTH_TOKEN) return "ANTHROPIC_AUTH_TOKEN"; + if (process.env.ANTHROPIC_API_KEY) return "ANTHROPIC_API_KEY"; + } + if (providerID === "openai" && process.env.OPENAI_API_KEY) return "OPENAI_API_KEY"; + return undefined; +} + +function anthropicBaseUrlForOpenCode(baseURL: string | undefined): string | undefined { + if (!baseURL) return undefined; + const normalized = baseURL.replace(/\/+$/, ""); + return normalized.endsWith("/v1") ? normalized : `${normalized}/v1`; +} + +export function buildOpenCodeConfig(config: Record): Config { + const cfg = readConfig(config); + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const providerID = cfg.aiProvider ?? requested.providerID; + const model = `${providerID}/${requested.modelID}`; + const variant = resolveOpenCodeVariant(providerID, cfg.thinkingLevel); + const maxTurns = cfg.maxTurns ?? DEFAULT_MAX_TURNS; + const credentialEnv = providerCredentialEnv(providerID, cfg); + const baseURL = + cfg.aiBaseUrl ?? + (providerID === "anthropic" + ? anthropicBaseUrlForOpenCode(process.env.ANTHROPIC_BASE_URL) + : providerID === "openai" + ? process.env.OPENAI_BASE_URL + : undefined); + + const providerOptions: Record = {}; + if (credentialEnv) providerOptions.apiKey = `{env:${credentialEnv}}`; + if (baseURL) providerOptions.baseURL = baseURL; + + const provider = + Object.keys(providerOptions).length > 0 || cfg.aiHeaders + ? { + [providerID]: { + ...(Object.keys(providerOptions).length > 0 ? { options: providerOptions } : {}), + ...(cfg.aiHeaders ? { headers: cfg.aiHeaders } : {}), + }, + } + : undefined; + + return { + // No logLevel: the v1 runtime accepts "WARN" and the v2 runtime accepts + // lowercase "warn" — the two CLIs disagree, and the default level is + // quiet enough that non-matching startup lines are simply ignored. + autoupdate: false, + share: "disabled", + snapshot: false, + formatter: false, + lsp: false, + plugin: [], + instructions: [], + mcp: {}, + model, + default_agent: MAIN_AGENT, + permission: READ_ONLY_PERMISSION, + tools: READ_ONLY_TOOLS, + ...(provider ? { provider } : {}), + agent: { + [MAIN_AGENT]: { + description: "Read-only static security analysis for deepsec", + mode: "primary", + model, + ...(variant ? { variant } : {}), + steps: maxTurns, + prompt: DEEPSEC_SYSTEM_NOTE, + tools: READ_ONLY_TOOLS, + permission: READ_ONLY_PERMISSION, + }, + [JSON_AGENT]: { + description: "Tool-free JSON formatting follow-up for deepsec", + mode: "primary", + hidden: true, + model, + ...(variant ? { variant } : {}), + steps: 1, + prompt: "Return only the JSON value requested by the user. Do not use tools.", + tools: NO_TOOLS, + permission: "deny", + }, + }, + }; +} + +/** + * OpenCode v2 permission rules: deny everything, then allow the read-only + * tool set. v2 evaluates ordered rules with the last match winning, so the + * catch-all deny comes first and the read-only allows come after. + * Validated live against OpenCode v2.0.24: bash/edit are denied while + * read/glob/grep/list still work. + */ +const V2_READ_ONLY_PERMISSIONS = [ + { action: "*", resource: "*", effect: "deny" }, + { action: "read", resource: "*", effect: "allow" }, + { action: "glob", resource: "*", effect: "allow" }, + { action: "grep", resource: "*", effect: "allow" }, + { action: "list", resource: "*", effect: "allow" }, +] as const; + +const V2_DENY_ALL_PERMISSIONS = [{ action: "*", resource: "*", effect: "deny" }] as const; + +/** + * Config content for the OpenCode v2 runtime (2.0.x). v2 renamed the v1 + * fields: `agent` → `agents`, `permission`/`tools` → `permissions` + * (ordered {action, resource, effect} rules), agent `prompt` → `system`, + * `autoupdate` → `update`. Structured output formats do not exist in v2; + * results arrive as plain text and go through the shared JSON-repair + * pipeline. + */ +export function buildOpenCodeV2Config(config: Record): Record { + const cfg = readConfig(config); + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const providerID = cfg.aiProvider ?? requested.providerID; + const model = `${providerID}/${requested.modelID}`; + const maxTurns = cfg.maxTurns ?? DEFAULT_MAX_TURNS; + const providerOverlays = openCodeV2ProviderOverlays(cfg); + + return { + share: "disabled", + update: "disable", + snapshots: false, + model, + default_agent: MAIN_AGENT, + permissions: V2_READ_ONLY_PERMISSIONS, + ...(Object.keys(providerOverlays).length > 0 ? { providers: providerOverlays } : {}), + agents: { + [MAIN_AGENT]: { + description: "Read-only static security analysis for deepsec", + mode: "primary", + system: DEEPSEC_SYSTEM_NOTE, + steps: maxTurns, + permissions: V2_READ_ONLY_PERMISSIONS, + }, + [JSON_AGENT]: { + description: "Tool-free JSON formatting follow-up for deepsec", + mode: "primary", + hidden: true, + system: "Return only the JSON value requested by the user. Do not use tools.", + steps: 1, + permissions: V2_DENY_ALL_PERMISSIONS, + }, + }, + }; +} + +/** + * v2 provider config overlays (baseURL redirects). The v2 built-in + * anthropic and openai providers take their base URL from config, not from + * the ANTHROPIC_BASE_URL / OPENAI_BASE_URL env vars deepsec's Gateway + * expansion sets — so the expansion is translated into the matching + * `providers..settings.baseURL` overlay, and a custom provider route + * (--ai-base-url) redirects the model's own provider the same way. + */ +export function openCodeV2ProviderOverlays( + cfg: OpenCodeAgentConfig, +): Record { + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const overlays: Record = {}; + + if (cfg.aiBaseUrl) { + // Custom route: redirect the model's own provider at the custom gateway. + overlays[requested.providerID] = { settings: { baseURL: cfg.aiBaseUrl } }; + return overlays; + } + + const anthropicBase = process.env.ANTHROPIC_BASE_URL; + if (anthropicBase) { + overlays.anthropic = { settings: { baseURL: anthropicBase } }; + } + const openaiBase = process.env.OPENAI_BASE_URL; + if (openaiBase) { + overlays.openai = { settings: { baseURL: openaiBase } }; + } + return overlays; +} + +/** + * Resolve extra environment variables the spawned OpenCode server should + * receive. v2's built-in providers read their credentials from the ambient + * environment (ANTHROPIC_ and OPENAI_ variables), so gateway and direct + * credentials flow through automatically. A custom provider override + * (--ai-provider + --ai-base-url + --ai-api-key-env) is remapped onto the + * matching standard provider pair, which v2's built-in providers honor. + */ +export function openCodeServerEnv( + cfg: OpenCodeAgentConfig, + protocol: OpenCodeProtocol, +): NodeJS.ProcessEnv { + if (protocol !== "v2") return {}; + if (!cfg.aiBaseUrl || !cfg.aiApiKeyEnv) { + // Gateway route: deepsec's startup expansion sets ANTHROPIC_AUTH_TOKEN + // (bearer) plus ANTHROPIC_BASE_URL. The v2 anthropic provider only reads + // ANTHROPIC_API_KEY (x-api-key) from the environment, so bridge the + // gateway token across when the API-key form is absent. + if ( + process.env.ANTHROPIC_BASE_URL && + !process.env.ANTHROPIC_API_KEY && + process.env.ANTHROPIC_AUTH_TOKEN + ) { + return { ANTHROPIC_API_KEY: process.env.ANTHROPIC_AUTH_TOKEN }; + } + return {}; + } + const credential = process.env[cfg.aiApiKeyEnv]; + if (!credential) { + throw new Error( + `OpenCode custom provider route requires ${cfg.aiApiKeyEnv}, but it is not set.`, + ); + } + if (cfg.aiHeaders && Object.keys(cfg.aiHeaders).length > 0) { + throw new Error( + "--ai-header overrides are not supported by the OpenCode v2 runtime; " + + "use the v1 opencode runtime or configure the provider headers in your opencode config.", + ); + } + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const providerID = cfg.aiProvider ?? requested.providerID; + if (providerID === "anthropic") { + return { ANTHROPIC_API_KEY: credential, ANTHROPIC_BASE_URL: cfg.aiBaseUrl }; + } + return { OPENAI_API_KEY: credential, OPENAI_BASE_URL: cfg.aiBaseUrl }; +} + +let cachedProtocolDetection: Promise | undefined; + +/** + * Detect the OpenCode runtime generation from `opencode --version`. v2 + * reports "opencode v2.0.24"; v1 reports "1.18.35". Cached per process — + * the binary on PATH does not change mid-run. Falls back to v1 when the + * version cannot be parsed (the legacy protocol is the conservative + * default: it is what the bundled sandbox runtime speaks). + */ +/** + * Map `opencode --version` output onto the runtime protocol. v2 reports + * "opencode v2.0.24"; v1 reports "1.18.35". Anything unparsable (or missing + * `opencode` entirely) falls back to v1 — the legacy protocol is what the + * bundled sandbox runtime speaks, making it the conservative default. + */ +export function protocolFromVersionOutput(output: string): OpenCodeProtocol { + const match = /\bv?(\d+)\.\d+\.\d+/.exec(output.trim()); + return match && Number(match[1]) >= 2 ? "v2" : "v1"; +} + +async function detectOpenCodeProtocol(): Promise { + cachedProtocolDetection ??= new Promise((resolve) => { + const child = spawn("opencode", ["--version"], { stdio: ["ignore", "pipe", "pipe"] }); + let output = ""; + const collect = (chunk: Buffer | string) => { + output += chunk.toString(); + }; + child.stdout?.on("data", collect); + child.stderr?.on("data", collect); + const finish = (protocol: OpenCodeProtocol) => { + cachedProtocolDetection = Promise.resolve(protocol); + resolve(protocol); + }; + child.on("error", () => finish("v1")); + child.on("exit", () => finish(protocolFromVersionOutput(output))); + }); + return cachedProtocolDetection; +} + +/** + * Spawn a private OpenCode server. Own spawn instead of the SDK's + * `createOpencodeServer`, which only recognizes the v1 startup line + * ("opencode server listening on …") and has no way to pass the password + * OpenCode v2 always requires. v2 generates an un guessable random password + * when none is provided — we generate one ourselves so the client can + * authenticate, and set both OPENCODE_PASSWORD (v2's preferred name) and + * OPENCODE_SERVER_PASSWORD (v1's). + */ +function spawnOpenCodeServer(params: { + port: number; + password: string; + configContent: string; + extraEnv?: NodeJS.ProcessEnv; + signal?: AbortSignal; + onProgress?: (progress: AgentProgress) => void; +}): Promise<{ url: string; close(): void }> { + return new Promise((resolve, reject) => { + const child = spawn("opencode", ["serve", `--hostname=127.0.0.1`, `--port=${params.port}`], { + env: { + ...process.env, + ...params.extraEnv, + OPENCODE_PASSWORD: params.password, + OPENCODE_SERVER_PASSWORD: params.password, + OPENCODE_CONFIG_CONTENT: params.configContent, + }, + stdio: ["ignore", "pipe", "pipe"], + }); + + let settled = false; + let output = ""; + const onOutput = (chunk: Buffer | string) => { + output += chunk.toString(); + if (settled) return; + for (const line of output.split("\n")) { + // v1 prints "opencode server listening on "; + // OpenCode v2 prints "server listening on ". + const match = /(?:^|\s)server listening on (https?:\/\/\S+)/.exec(line); + if (match) { + settled = true; + cleanup(); + resolve({ url: match[1], close: () => child.kill() }); + return; + } + } + }; + const timer = setTimeout(() => { + if (settled) return; + settled = true; + cleanup(); + child.kill(); + reject( + new Error( + `Timeout waiting for the OpenCode server to start after 60s.\n` + + `Server output: ${output.trim().slice(0, 500) || "(none)"}`, + ), + ); + }, 60_000); + const onExit = (code: number | null) => { + if (settled) return; + settled = true; + cleanup(); + reject( + new Error( + `OpenCode server exited with code ${code}${output.trim() ? `\nServer output: ${output.trim().slice(0, 500)}` : ""}`, + ), + ); + }; + const onError = (error: Error) => { + if (settled) return; + settled = true; + cleanup(); + reject( + new Error( + `Failed to spawn the OpenCode server. Is \`opencode\` installed and on PATH? (${error.message})`, + ), + ); + }; + const onAbort = () => { + if (settled) return; + settled = true; + cleanup(); + child.kill(); + reject(params.signal?.reason ?? new Error("OpenCode server startup aborted")); + }; + function cleanup() { + clearTimeout(timer); + child.stdout?.off("data", onOutput); + child.stderr?.off("data", onOutput); + child.off("exit", onExit); + child.off("error", onError); + params.signal?.removeEventListener("abort", onAbort); + } + child.stdout?.on("data", onOutput); + child.stderr?.on("data", onOutput); + child.on("exit", onExit); + child.on("error", onError); + if (params.signal?.aborted) onAbort(); + else params.signal?.addEventListener("abort", onAbort, { once: true }); + }); +} + +async function findFreePort(): Promise { + return await new Promise((resolve, reject) => { + const server = net.createServer(); + server.unref(); + server.once("error", reject); + server.listen(0, "127.0.0.1", () => { + const address = server.address(); + if (!address || typeof address === "string") { + server.close(); + reject(new Error("OpenCode could not allocate a local server port")); + return; + } + const port = address.port; + server.close((err) => { + if (err) reject(err); + else resolve(port); + }); + }); + }); +} + +function createOpenCodeFetch(dispatcher: UndiciAgent, authHeader: string): typeof globalThis.fetch { + return (async (input: RequestInfo | URL, init?: RequestInit) => { + // The generated SDK constructs Node's built-in Request, while the + // separately versioned Undici package has its own branded Request class. + // Normalize to primitives so fetch and its dispatcher always come from + // the same Undici version. The injected authorization header serves the + // password-secured server (v1 with OPENCODE_SERVER_PASSWORD set, v2 + // always). + const request = input instanceof Request ? input : new Request(input, init); + const body = + request.method === "GET" || request.method === "HEAD" + ? undefined + : new Uint8Array(await request.arrayBuffer()); + return (await undiciFetch(request.url, { + method: request.method, + headers: { + ...Object.fromEntries(request.headers.entries()), + authorization: authHeader, + }, + body, + redirect: request.redirect, + signal: request.signal, + dispatcher, + })) as unknown as Response; + }) as typeof globalThis.fetch; +} + +async function createRunContext(params: { + projectRoot: string; + config: Record; + signal?: AbortSignal; + title: string; +}): Promise { + const controller = new AbortController(); + const onParentAbort = () => + controller.abort(params.signal?.reason ?? new Error("OpenCode run aborted")); + if (params.signal) { + if (params.signal.aborted) onParentAbort(); + else params.signal.addEventListener("abort", onParentAbort, { once: true }); + } + const detachParentAbort = () => params.signal?.removeEventListener("abort", onParentAbort); + + let server: { url: string; close(): void } | undefined; + const dispatcher = new UndiciAgent({ + // Prompts are long-running: OpenCode does not send response headers + // until the entire agent loop finishes (v1) or the inbox wait resolves + // (v2). Undici's 300-second defaults turn valid long-running batches + // into a misleading `fetch failed`. + headersTimeout: 0, + bodyTimeout: 0, + }); + try { + const cfg = readConfig(params.config); + const protocol = await detectOpenCodeProtocol(); + // One password, two env names: v2 reads OPENCODE_PASSWORD first and + // v1 reads OPENCODE_SERVER_PASSWORD. Both secure the server with basic + // auth (username "opencode"), and v2 *always* requires it — without a + // known value it generates an un guessable random one. + const password = crypto.randomBytes(24).toString("base64url"); + const authHeader = `Basic ${Buffer.from(`opencode:${password}`).toString("base64")}`; + + const port = await findFreePort(); + server = await spawnOpenCodeServer({ + port, + password, + signal: controller.signal, + configContent: JSON.stringify( + protocol === "v2" + ? buildOpenCodeV2Config(params.config) + : buildOpenCodeConfig(params.config), + ), + extraEnv: openCodeServerEnv(cfg, protocol), + }); + + // Authoritative protocol check: the /api/info endpoint only exists on + // the v2 runtime. Trust it over the version probe — the config content + // was already chosen, but a mismatch surfaces here as a clear error + // instead of cryptic per-request 404/405s. + const isV2 = await probeV2Server(server.url, authHeader, dispatcher, controller.signal); + if (isV2 !== (protocol === "v2")) { + throw new Error( + `OpenCode version reported ${protocol === "v2" ? "v2" : "v1"}, but the server ` + + `exposes the ${isV2 ? "v2 (/api)" : "v1 (legacy)"} API. The config content sent at ` + + `startup matches the reported version; restart with a matching opencode runtime.`, + ); + } + + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const providerID = cfg.aiProvider ?? requested.providerID; + const model = { + providerID, + modelID: requested.modelID, + }; + // v1 maps the thinking dial onto provider variants (anthropic: max/high); + // v2 keeps its model default unless a level was explicitly requested. + const variant = + protocol === "v2" + ? cfg.thinkingLevel + ? resolveOpenCodeVariant(providerID, cfg.thinkingLevel) + : undefined + : resolveOpenCodeVariant(providerID, cfg.thinkingLevel); + + if (protocol === "v2") { + const created = await v2Call( + server.url, + authHeader, + dispatcher, + controller.signal, + "POST", + "/api/session", + { + location: { directory: params.projectRoot }, + title: params.title, + agent: MAIN_AGENT, + model: { + providerID: model.providerID, + id: model.modelID, + ...(variant ? { variant } : {}), + }, + }, + ); + const sessionID = created?.data?.id; + if (typeof sessionID !== "string") { + throw new Error( + `OpenCode v2 session creation returned no session id: ${JSON.stringify(created).slice(0, 300)}`, + ); + } + return { + protocol, + client: undefined, + server, + dispatcher, + authHeader, + sessionID, + directory: params.projectRoot, + model, + controller, + detachParentAbort, + }; + } + + const client = createOpencodeClient({ + baseUrl: server.url, + fetch: createOpenCodeFetch(dispatcher, authHeader), + }); + const created = await client.session.create( + { + directory: params.projectRoot, + title: params.title, + agent: MAIN_AGENT, + model: { + providerID, + id: requested.modelID, + ...(variant ? { variant } : {}), + }, + }, + { throwOnError: true, signal: controller.signal }, + ); + + return { + protocol, + client, + server, + dispatcher, + authHeader, + sessionID: created.data.id, + directory: params.projectRoot, + model, + controller, + detachParentAbort, + }; + } catch (err) { + server?.close(); + await dispatcher.close().catch(() => undefined); + detachParentAbort(); + throw err; + } +} + +async function disposeRunContext(context: OpenCodeRunContext | undefined): Promise { + if (!context) return; + context.detachParentAbort(); + if (!context.controller.signal.aborted) { + try { + if (context.protocol === "v2") { + await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "DELETE", + `/api/session/${context.sessionID}`, + ); + } else { + await context.client?.session.delete( + { sessionID: context.sessionID, directory: context.directory }, + { throwOnError: true }, + ); + } + } catch { + // Best effort: server shutdown is the authoritative cleanup. + } + } + context.server.close(); + await context.dispatcher.close(); +} + +/** Minimal JSON shape of a v2 assistant message (Session.Message.Assistant). */ +interface V2AssistantMessage { + id?: string; + type: "assistant"; + agent?: string; + model?: { id?: string; providerID?: string; variant?: string }; + content?: Array< + | { type: "text"; text: string } + | { type: "reasoning"; text: string } + | { + type: "tool"; + name?: string; + state?: { status?: string; error?: string }; + } + >; + cost?: number; + tokens?: { + input?: number; + output?: number; + reasoning?: number; + cache?: { read?: number; write?: number }; + }; + error?: { message?: string } & Record; + retry?: { attempt?: number; error?: { message?: string } }; + time?: { created?: number; completed?: number }; +} + +/** + * One request against the OpenCode v2 server. All routes live under /api, + * require basic auth, and return `{ data: … }` envelopes (or 204 No Content). + */ +async function v2Call( + baseUrl: string, + authHeader: string, + dispatcher: UndiciAgent, + signal: AbortSignal | undefined, + method: string, + path: string, + body?: unknown, +): Promise { + const response = await undiciFetch(`${baseUrl}${path}`, { + method, + headers: { + authorization: authHeader, + accept: "application/json", + ...(body === undefined ? {} : { "content-type": "application/json" }), + }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + signal, + dispatcher, + }); + const text = await response.text(); + if (!response.ok) { + throw new Error( + `OpenCode v2 ${method} ${path} failed (HTTP ${response.status}): ${text.slice(0, 300) || "(empty body)"}`, + ); + } + if (response.status === 204 || text === "") return undefined; + try { + return JSON.parse(text); + } catch (err) { + throw new Error( + `OpenCode v2 ${method} ${path} returned non-JSON output: ${text.slice(0, 300)}`, + { cause: err }, + ); + } +} + +/** + * Probe for the v2 runtime: `GET /api/info` exists only on the v2 server and + * returns `{version, pid, …}`. The v1 runtime falls through to its static + * web handler (HTML) or 404s. + */ +async function probeV2Server( + baseUrl: string, + authHeader: string, + dispatcher: UndiciAgent, + signal: AbortSignal | undefined, +): Promise { + try { + const response = await undiciFetch(`${baseUrl}/api/info`, { + headers: { authorization: authHeader, accept: "application/json" }, + signal, + dispatcher, + }); + if (!response.ok) return false; + const body: unknown = await response.json(); + return ( + typeof body === "object" && + body !== null && + typeof (body as { version?: unknown }).version === "string" + ); + } catch { + return false; + } +} + +/** + * Read the assistant messages produced after a given user message, newest + * first (the v2 message list is `order=desc`). Stops at the user message + * that started the cycle; if it is not on the first page, every assistant + * message found is kept (defensive against long tool-driven cycles). + */ +async function v2AssistantMessagesSince( + context: OpenCodeRunContext, + userMessageID: string | undefined, +): Promise { + const list = await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "GET", + `/api/session/${context.sessionID}/message?order=desc&limit=200`, + ); + const data: Array<{ id?: string; type?: string }> = Array.isArray(list?.data) ? list.data : []; + const collected: V2AssistantMessage[] = []; + for (const message of data) { + if (userMessageID && message.id === userMessageID) break; + if (message.type === "assistant") { + collected.push(message as V2AssistantMessage); + } + } + return collected; +} + +function v2ProgressFromAssistant(message: V2AssistantMessage): AgentProgress[] { + const progress: AgentProgress[] = []; + for (const part of message.content ?? []) { + if (part.type === "tool") { + progress.push({ + type: part.state?.status === "error" ? "error" : "tool_use", + message: + part.state?.status === "error" + ? `OpenCode ${part.name ?? "tool"} error: ${(part.state.error ?? "").slice(0, 300)}` + : `${part.name ?? "tool"}`, + }); + } + } + if (message.retry) { + progress.push({ + type: "thinking", + message: `OpenCode provider retry ${message.retry.attempt ?? "?"}: ${(message.retry.error?.message ?? "").slice(0, 200)}`, + }); + } + return progress; +} + +/** Run one prompt cycle against the v2 async inbox: prompt → wait → read. */ +async function runV2Prompt(params: { + context: OpenCodeRunContext; + prompt: string; +}): Promise { + const context = params.context; + const startTime = Date.now(); + const posted = await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "POST", + `/api/session/${context.sessionID}/prompt`, + { text: params.prompt }, + ); + const userMessageID: string | undefined = posted?.data?.id; + // The inbox admits the prompt and schedules the agent loop; the wait + // endpoint resolves (204) once the loop goes idle. + await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "POST", + `/api/experimental/session/${context.sessionID}/wait`, + ); + const messages = await v2AssistantMessagesSince(context, userMessageID); + + const progress: AgentProgress[] = []; + let toolUseCount = 0; + let costUsd: number | undefined; + const usage = { + inputTokens: 0, + outputTokens: 0, + cacheReadInputTokens: 0, + cacheCreationInputTokens: 0, + }; + let usageSeen = false; + for (const message of messages) { + for (const item of v2ProgressFromAssistant(message)) progress.push(item); + if (message.error) { + const detail = message.error.message ?? JSON.stringify(message.error).slice(0, 300); + throw new Error(`OpenCode v2 assistant error: ${detail}`); + } + toolUseCount += (message.content ?? []).filter((part) => part.type === "tool").length; + if (typeof message.cost === "number") { + costUsd = (costUsd ?? 0) + message.cost; + } + const tokens = message.tokens; + if (tokens) { + usageSeen = true; + usage.inputTokens += tokens.input ?? 0; + usage.outputTokens += tokens.output ?? 0; + usage.cacheReadInputTokens += tokens.cache?.read ?? 0; + usage.cacheCreationInputTokens += tokens.cache?.write ?? 0; + } + } + + // Newest-first: the final answer is the most recent assistant text. + let resultText = ""; + for (const message of messages) { + const text = (message.content ?? []) + .filter((part): part is { type: "text"; text: string } => part.type === "text") + .map((part) => part.text) + .join("\n") + .trim(); + if (text) { + resultText = text; + break; + } + } + + return { + resultText, + turnCount: Math.max(1, messages.length), + toolUseCount, + progress, + meta: { + durationApiMs: Date.now() - startTime, + numTurns: Math.max(1, messages.length), + ...(costUsd !== undefined ? { costUsd } : {}), + agentSessionId: context.sessionID, + ...(usageSeen ? { usage } : {}), + durationMs: Date.now() - startTime, + }, + }; +} + +function formatAssistantError(error: NonNullable): string { + const message = + error.data && typeof error.data === "object" && "message" in error.data + ? String(error.data.message) + : JSON.stringify(error.data); + return `${error.name}: ${message}`; +} + +function assistantError(error: NonNullable): Error { + const result = new Error(formatAssistantError(error)); + result.name = error.name; + return result; +} + +function isStructuredOutputError(error: unknown): boolean { + return error instanceof Error && error.name === "StructuredOutputError"; +} + +export function formatOpenCodeError(error: unknown): string { + if (!(error instanceof Error)) return String(error); + + const messages = [error.message]; + let cause: unknown = error.cause; + const seen = new Set([error]); + while (cause instanceof Error && !seen.has(cause)) { + seen.add(cause); + const code = + "code" in cause && typeof cause.code === "string" && cause.code ? ` (${cause.code})` : ""; + const message = `${cause.message}${code}`; + if (!messages.includes(message)) messages.push(message); + cause = cause.cause; + } + return messages.join(": "); +} + +export function resolveOpenCodeAssistantText( + info: Pick, + partText: string, +): OpenCodeResolvedText { + if (info.structured !== undefined) { + return { + resultText: JSON.stringify(info.structured), + recoveredStructuredText: false, + }; + } + if (!info.error) { + return { + resultText: partText, + recoveredStructuredText: false, + }; + } + if (info.error.name === "StructuredOutputError" && partText) { + return { + resultText: partText, + recoveredStructuredText: true, + }; + } + throw assistantError(info.error); +} + +function textFromParts(parts: Part[]): string { + const text: string[] = []; + for (const part of parts) { + if (part.type === "text" && !part.ignored) text.push(part.text); + } + return text.join("\n").trim(); +} + +function shortToolTarget(part: Extract): string | undefined { + const input = part.state.input; + for (const key of ["path", "filePath", "file_path", "pattern", "query"]) { + const value = input[key]; + if (typeof value === "string" && value.trim()) { + return value.split("/").slice(-3).join("/"); + } + } + return undefined; +} + +function progressFromParts(parts: Part[]): AgentProgress[] { + const progress: AgentProgress[] = []; + for (const part of parts) { + if (part.type === "tool") { + const target = shortToolTarget(part); + progress.push({ + type: part.state.status === "error" ? "error" : "tool_use", + message: + part.state.status === "error" + ? `OpenCode ${part.tool} error${target ? `: ${target}` : ""}: ${part.state.error.slice(0, 300)}` + : `${part.tool}${target ? `: ${target}` : ""}`, + candidateFile: target, + }); + } else if (part.type === "retry") { + progress.push({ + type: "thinking", + message: `OpenCode provider retry ${part.attempt}: ${part.error.data.message.slice(0, 200)}`, + }); + } else if (part.type === "compaction") { + progress.push({ type: "thinking", message: "OpenCode compacted conversation context" }); + } + } + return progress; +} + +async function runPrompt(params: { + context: OpenCodeRunContext; + prompt: string; + format: OutputFormat; + config: Record; + tools?: Record; + agent?: string; +}): Promise { + // v2 has no structured-output prompt formats; its results arrive as plain + // text and the shared JSON-repair pipeline validates them. + if (params.context.protocol === "v2") { + return await runV2Prompt({ context: params.context, prompt: params.prompt }); + } + const cfg = readConfig(params.config); + const requested = parseOpenCodeModel(cfg.model ?? DEFAULT_MODEL); + const providerID = cfg.aiProvider ?? requested.providerID; + const variant = resolveOpenCodeVariant(providerID, cfg.thinkingLevel); + const useTextFormat = shouldUseOpenCodeTextFormat(providerID, variant, params.format); + const startTime = Date.now(); + + const response = await params.context.client!.session.prompt( + { + sessionID: params.context.sessionID, + directory: params.context.directory, + model: { providerID, modelID: requested.modelID }, + agent: params.agent ?? MAIN_AGENT, + ...(variant ? { variant } : {}), + tools: params.tools ?? READ_ONLY_TOOLS, + format: useTextFormat ? TEXT_FORMAT : params.format, + parts: [{ type: "text", text: params.prompt }], + }, + { throwOnError: true, signal: params.context.controller.signal }, + ); + + const { info, parts } = response.data; + const resolved = resolveOpenCodeAssistantText(info, textFromParts(parts)); + const turnCount = Math.max(1, parts.filter((part) => part.type === "step-finish").length); + const toolUseCount = parts.filter((part) => part.type === "tool").length; + const completed = info.time.completed ?? Date.now(); + const progress = progressFromParts(parts); + if (useTextFormat) { + progress.push({ + type: "thinking", + message: + "Anthropic thinking is incompatible with forced StructuredOutput; validating the text response", + }); + } + if (resolved.recoveredStructuredText) { + progress.push({ + type: "thinking", + message: + "OpenCode returned text instead of calling StructuredOutput; validating the text response", + }); + } + + return { + resultText: resolved.resultText, + turnCount, + toolUseCount, + progress, + meta: { + durationApiMs: Math.max(0, completed - info.time.created), + numTurns: turnCount, + costUsd: info.cost, + agentSessionId: info.sessionID, + usage: { + inputTokens: info.tokens.input, + outputTokens: info.tokens.output, + cacheReadInputTokens: info.tokens.cache.read, + cacheCreationInputTokens: info.tokens.cache.write, + }, + durationMs: Date.now() - startTime, + }, + }; +} + +async function runToollessFollowUp(params: { + context: OpenCodeRunContext | undefined; + prompt: string; + config: Record; +}): Promise { + if (!params.context) return undefined; + const context = params.context; + if (context.protocol === "v2") { + // v2 prompts cannot carry a per-request agent override; switch the + // session to the deny-all deepsec-json agent for the follow-up and + // switch back afterwards. + try { + const switched = await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "POST", + `/api/session/${context.sessionID}/agent`, + { agent: JSON_AGENT }, + ); + const result = await runV2Prompt({ context, prompt: params.prompt }); + if (!switched) { + await v2Call( + context.server.url, + context.authHeader, + context.dispatcher, + context.controller.signal, + "POST", + `/api/session/${context.sessionID}/agent`, + { agent: MAIN_AGENT }, + ); + } + return result.resultText || undefined; + } catch { + return undefined; + } + } + try { + const result = await runPrompt({ + context, + prompt: params.prompt, + format: TEXT_FORMAT, + config: params.config, + tools: NO_TOOLS, + agent: JSON_AGENT, + }); + return result.resultText; + } catch { + return undefined; + } +} + +async function runRefusalFollowUp(params: { + context: OpenCodeRunContext | undefined; + config: Record; +}): Promise { + const raw = await runToollessFollowUp({ + ...params, + prompt: REFUSAL_FOLLOWUP_PROMPT, + }); + return raw === undefined ? undefined : parseRefusalReport(raw); +} + +export class OpenCodeAgentPlugin implements AgentPlugin { + type = "opencode"; + + async *investigate(params: InvestigateParams): AsyncGenerator { + const { batch, projectRoot, promptTemplate, projectInfo, config, signal, projectId } = params; + const prompt = buildInvestigatePrompt({ promptTemplate, projectInfo, batch }); + const model = readConfig(config).model ?? DEFAULT_MODEL; + const startTime = Date.now(); + let context: OpenCodeRunContext | undefined; + let resultText = ""; + let lastError = ""; + let sdkMeta: Partial = {}; + let turnCount = 0; + let toolUseCount = 0; + let attempts = 0; + + try { + for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) { + attempts = attempt; + if (attempt > 1) { + yield { + type: "thinking", + message: `Retrying OpenCode batch after transient error (attempt ${attempt}/${MAX_ATTEMPTS}): ${lastError.slice(0, 200)}`, + }; + await disposeRunContext(context); + context = undefined; + resultText = ""; + lastError = ""; + sdkMeta = {}; + turnCount = 0; + toolUseCount = 0; + } + + try { + context = await createRunContext({ + projectRoot, + config, + signal, + title: `deepsec investigate (${batch.length} files)`, + }); + if (attempt === 1) { + yield { + type: "started", + message: `Investigating ${batch.length} file(s) with OpenCode (${model})`, + }; + } + const run = await runPrompt({ + context, + prompt, + format: INVESTIGATE_FORMAT, + config, + }); + resultText = run.resultText; + sdkMeta = run.meta; + turnCount = run.turnCount; + toolUseCount = run.toolUseCount; + for (const progress of run.progress) yield progress; + } catch (err) { + if (isStructuredOutputError(err)) { + yield { + type: "thinking", + message: + "OpenCode did not call StructuredOutput; requesting a tool-free JSON finalization", + }; + const recovered = await runToollessFollowUp({ + context, + prompt: buildInvestigateJsonRepairPrompt(batch), + config, + }); + if (recovered) resultText = recovered; + } + if (!resultText) { + lastError = formatOpenCodeError(err); + yield { type: "error", message: `OpenCode SDK error: ${lastError.slice(0, 300)}` }; + } + } + + if (resultText) break; + const quotaSource = classifyQuotaError(lastError); + if (quotaSource) throw new QuotaExhaustedError(quotaSource, lastError); + if (attempt >= MAX_ATTEMPTS || !isTransientError(lastError)) break; + await backoff(attempt); + } + + if (!resultText) { + throw new Error( + `OpenCode produced no investigation result after ${attempts} attempt(s). ` + + `Last error: ${lastError || "(none captured)"}.`, + ); + } + + let parsedOutcome: ParsedInvestigateResults; + try { + parsedOutcome = parseInvestigateResults(resultText, batch); + } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } + yield { + type: "thinking", + message: "OpenCode returned non-JSON investigation output; requesting JSON-only repair", + }; + const repairText = await runToollessFollowUp({ + context, + prompt: buildInvestigateJsonRepairPrompt(batch), + config, + }); + if (repairText === undefined) { + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } + try { + parsedOutcome = parseInvestigateResults(repairText, batch); + resultText = repairText; + yield { type: "thinking", message: "OpenCode JSON repair succeeded" }; + } catch (repairErr) { + const combinedError = jsonRepairFailureError(err, repairErr); + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText: formatJsonRepairFailureDebugText(resultText, repairText), + error: combinedError, + batch, + }); + throw combinedError; + } + } + let results: InvestigateResult[] = parsedOutcome.results; + if (parsedOutcome.invalid.length > 0) { + const fieldRepair = yield* runInvestigateFieldRepairLoop({ + results, + invalid: parsedOutcome.invalid, + batch, + followUp: (prompt) => runToollessFollowUp({ context, prompt, config }), + agentLabel: "OpenCode", + agentType: this.type, + projectId, + }); + results = fieldRepair.results; + } + + const refusal = await runRefusalFollowUp({ context, config }); + if (refusal?.refused) { + yield { + type: "thinking", + message: `Refusal detected: ${refusal.reason ?? "see raw"}`, + }; + } + + const durationMs = Date.now() - startTime; + const costStr = sdkMeta.costUsd != null ? ` $${sdkMeta.costUsd.toFixed(3)}` : ""; + const tokensStr = sdkMeta.usage + ? ` ${sdkMeta.usage.inputTokens + sdkMeta.usage.outputTokens} tokens` + : ""; + yield { + type: "complete", + message: `Investigation complete (${(durationMs / 1000).toFixed(1)}s, ${turnCount} turns, ${toolUseCount} tool calls${costStr}${tokensStr}${refusal?.refused ? " refusal" : ""})`, + }; + + return { + results, + meta: { + durationMs, + ...sdkMeta, + refusal, + }, + }; + } finally { + await disposeRunContext(context); + } + } + + async *revalidate(params: RevalidateParams): AsyncGenerator { + const { batch, projectRoot, projectInfo, config, force = false, signal, projectId } = params; + const { prompt, totalFindings } = buildRevalidatePrompt({ + batch, + projectRoot, + projectInfo, + force, + }); + const model = readConfig(config).model ?? DEFAULT_MODEL; + const startTime = Date.now(); + let context: OpenCodeRunContext | undefined; + let resultText = ""; + let lastError = ""; + let sdkMeta: Partial = {}; + let turnCount = 0; + let toolUseCount = 0; + let attempts = 0; + + try { + for (let attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) { + attempts = attempt; + if (attempt > 1) { + yield { + type: "thinking", + message: `Retrying OpenCode revalidation after transient error (attempt ${attempt}/${MAX_ATTEMPTS}): ${lastError.slice(0, 200)}`, + }; + await disposeRunContext(context); + context = undefined; + resultText = ""; + lastError = ""; + sdkMeta = {}; + turnCount = 0; + toolUseCount = 0; + } + + try { + context = await createRunContext({ + projectRoot, + config, + signal, + title: `deepsec revalidate (${totalFindings} findings)`, + }); + if (attempt === 1) { + yield { + type: "started", + message: `Revalidating ${totalFindings} finding(s) across ${batch.length} file(s) with OpenCode (${model})`, + }; + } + const run = await runPrompt({ + context, + prompt, + format: REVALIDATE_FORMAT, + config, + }); + resultText = run.resultText; + sdkMeta = run.meta; + turnCount = run.turnCount; + toolUseCount = run.toolUseCount; + for (const progress of run.progress) yield progress; + } catch (err) { + if (isStructuredOutputError(err)) { + yield { + type: "thinking", + message: + "OpenCode did not call StructuredOutput; requesting a tool-free JSON finalization", + }; + const recovered = await runToollessFollowUp({ + context, + prompt: buildRevalidateJsonRepairPrompt(), + config, + }); + if (recovered) resultText = recovered; + } + if (!resultText) { + lastError = formatOpenCodeError(err); + yield { type: "error", message: `OpenCode SDK error: ${lastError.slice(0, 300)}` }; + } + } + + if (resultText) break; + const quotaSource = classifyQuotaError(lastError); + if (quotaSource) throw new QuotaExhaustedError(quotaSource, lastError); + if (attempt >= MAX_ATTEMPTS || !isTransientError(lastError)) break; + await backoff(attempt); + } + + if (!resultText) { + throw new Error( + `OpenCode produced no revalidation result after ${attempts} attempt(s). ` + + `Last error: ${lastError || "(none captured)"}.`, + ); + } + + let verdicts: RevalidateVerdict[]; + try { + verdicts = parseRevalidateVerdicts(resultText); + } catch (err) { + yield { + type: "thinking", + message: "OpenCode returned non-JSON revalidation output; requesting JSON-only repair", + }; + const repairText = await runToollessFollowUp({ + context, + prompt: buildRevalidateJsonRepairPrompt(), + config, + }); + if (repairText === undefined) { + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: this.type, + resultText, + error: err, + batch, + }); + throw err; + } + try { + verdicts = parseRevalidateVerdicts(repairText); + resultText = repairText; + yield { type: "thinking", message: "OpenCode JSON repair succeeded" }; + } catch (repairErr) { + const combinedError = jsonRepairFailureError(err, repairErr); + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: this.type, + resultText: formatJsonRepairFailureDebugText(resultText, repairText), + error: combinedError, + batch, + }); + throw combinedError; + } + } + + const refusal = await runRefusalFollowUp({ context, config }); + if (refusal?.refused) { + yield { + type: "thinking", + message: `Refusal detected during revalidation: ${refusal.reason ?? "see raw"}`, + }; + } + + const durationMs = Date.now() - startTime; + const costStr = sdkMeta.costUsd != null ? ` $${sdkMeta.costUsd.toFixed(3)}` : ""; + yield { + type: "complete", + message: `Revalidation complete (${(durationMs / 1000).toFixed(1)}s, ${turnCount} turns, ${toolUseCount} tool calls${costStr}, ${verdicts.length} verdicts${refusal?.refused ? " refusal" : ""})`, + }; + + return { + verdicts, + meta: { durationMs, ...sdkMeta, refusal }, + }; + } finally { + await disposeRunContext(context); + } + } +} diff --git a/packages/processor/src/agents/pi-sdk.ts b/packages/processor/src/agents/pi-sdk.ts index fe9a0524..62cc6e8d 100644 --- a/packages/processor/src/agents/pi-sdk.ts +++ b/packages/processor/src/agents/pi-sdk.ts @@ -18,6 +18,7 @@ import { type ToolDefinition, } from "@earendil-works/pi-coding-agent"; import { + AgentPolicyRefusalError, attributionHeaders, backoff, buildInvestigateJsonRepairPrompt, @@ -965,6 +966,18 @@ export class PiAgentPlugin implements AgentPlugin { try { parsed = parseInvestigateResults(resultText, batch); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "investigate", + agentType: this.type, + resultText, + error: err, + batch, + }); + session?.dispose(); + throw err; + } yield { type: "thinking", message: "Pi returned non-JSON investigation output; requesting JSON-only repair", @@ -1142,6 +1155,18 @@ export class PiAgentPlugin implements AgentPlugin { try { verdicts = parseRevalidateVerdicts(resultText); } catch (err) { + if (err instanceof AgentPolicyRefusalError) { + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: this.type, + resultText, + error: err, + batch, + }); + session?.dispose(); + throw err; + } yield { type: "thinking", message: "Pi returned non-JSON revalidation output; requesting JSON-only repair", diff --git a/packages/processor/src/agents/shared.ts b/packages/processor/src/agents/shared.ts index 6e3bdb3f..63bf0b03 100644 --- a/packages/processor/src/agents/shared.ts +++ b/packages/processor/src/agents/shared.ts @@ -218,6 +218,74 @@ export class QuotaExhaustedError extends Error { } } +// --- Policy refusal -------------------------------------------------------- + +/** + * Hard provider refusals (safety policy), distinct from quota: the provider + * declined the request itself, so the agent never produced findings JSON. + * - `anthropic-usage-policy` — Claude Code usage-policy (AUP) block + * - `openai-safety-system` — OpenAI `invalid_prompt` / safety-system prose + * - `content-policy` — generic `content_policy_violation` envelope + */ +type RefusalSource = "anthropic-usage-policy" | "openai-safety-system" | "content-policy"; + +/** + * Classify agent output as a hard provider policy refusal, or `undefined`. + * + * Matching is intentionally strict — the opposite trade-off from + * `classifyQuotaError`: a false positive here would relabel ordinary + * malformed JSON as "the model declined" (zero coverage over the batch), + * which is worse than the status quo. Misses fall through to the existing + * fail-loud parse error, which stays accurate. + */ +export function classifyPolicyRefusal(msg: string): RefusalSource | undefined { + if (!msg) return undefined; + const m = msg.toLowerCase(); + + // Claude Code usage-policy block: + // "API Error: Claude Code is unable to respond to this request, which + // appears to violate our Usage Policy (https://www.anthropic.com/legal/aup)." + if ( + /anthropic\.com\/legal\/aup/.test(m) || + (/unable to respond to this request/.test(m) && /usage polic/.test(m)) + ) { + return "anthropic-usage-policy"; + } + + // OpenAI safety-system rejection: the `invalid_prompt` error code, or the + // prose form that accompanies it. + if (/\binvalid_prompt\b/.test(m) || /rejected as a result of our safety system/.test(m)) { + return "openai-safety-system"; + } + + if (/\bcontent_policy_violation\b/.test(m)) { + return "content-policy"; + } + + return undefined; +} + +/** + * Thrown when the provider refused the request on policy grounds instead of + * producing findings JSON. Unlike `QuotaExhaustedError` this does NOT abort + * the run: a refusal depends on the batch's content, so other batches can + * proceed — but this batch must surface as errored, and must skip the + * JSON-repair follow-up (which would otherwise ask a declining model to + * re-emit conclusions it never produced, and accepts an empty findings + * array as a valid answer). + */ +export class AgentPolicyRefusalError extends Error { + readonly source: RefusalSource; + readonly rawMessage: string; + + constructor(source: RefusalSource, rawMessage: string) { + super(`Agent refused the request on policy grounds (${source}): ${rawMessage.slice(0, 300)}`); + this.name = "AgentPolicyRefusalError"; + this.source = source; + this.rawMessage = rawMessage; + } +} + /** * Vercel AI Gateway endpoints — duplicated from preflight.ts on purpose: * processor doesn't depend on the deepsec CLI package. Keep these in sync @@ -733,7 +801,7 @@ function extractArrayCandidateForRepair(jsonPayload: string): string { return lastClose >= 0 ? candidate.slice(0, lastClose + 1) : candidate; } -function parseAgentJsonArray(params: { +export function parseAgentJsonArray(params: { resultText: string; parseFailurePrefix: string; nonArrayMessage: (parsed: unknown) => string; @@ -745,6 +813,14 @@ function parseAgentJsonArray(params: { try { parsed = JSON.parse(jsonPayload); } catch (strictErr) { + // Classify before attempting repair, so a refusal is never "repaired" + // into a valid-but-empty findings array. Doing it inside the catch — + // rather than before JSON.parse — means valid findings JSON that merely + // quotes refusal text as evidence parses normally. + const refusalSource = classifyPolicyRefusal(resultText); + if (refusalSource) { + throw new AgentPolicyRefusalError(refusalSource, resultText); + } const repairCandidate = extractArrayCandidateForRepair(jsonPayload); try { parsed = JSON.parse(jsonrepair(repairCandidate)); diff --git a/packages/processor/src/enrich.ts b/packages/processor/src/enrich.ts index d73b3bbe..5c4299ba 100644 --- a/packages/processor/src/enrich.ts +++ b/packages/processor/src/enrich.ts @@ -6,6 +6,7 @@ import { getRegistry, loadAllFileRecords, readProjectConfig, + SEVERITY_ORDER, writeFileRecord, } from "@deepsec/core"; @@ -153,15 +154,6 @@ interface EnrichProgress { total?: number; } -const SEVERITY_ORDER: Record = { - CRITICAL: 0, - HIGH: 1, - HIGH_BUG: 2, - MEDIUM: 3, - BUG: 4, - LOW: 5, -}; - export async function enrich(params: { projectId: string; filter?: string; diff --git a/packages/processor/src/index.ts b/packages/processor/src/index.ts index 09bd3923..a8e3f022 100644 --- a/packages/processor/src/index.ts +++ b/packages/processor/src/index.ts @@ -16,12 +16,14 @@ import { readProjectConfig, readRunMeta, registerActiveRun, + SEVERITY_ORDER, writeFileRecord, writeRunMeta, } from "@deepsec/core"; import { noiseScore, readTechJson } from "@deepsec/scanner"; import { ClaudeAgentSdkPlugin } from "./agents/claude-agent-sdk.js"; import { CodexAgentSdkPlugin } from "./agents/codex-sdk.js"; +import { OpenCodeAgentPlugin } from "./agents/opencode-sdk.js"; import { PiAgentPlugin } from "./agents/pi-sdk.js"; import { AgentRegistry } from "./agents/registry.js"; import { QuotaExhaustedError, type QuotaSource } from "./agents/shared.js"; @@ -45,6 +47,7 @@ import { export { ClaudeAgentSdkPlugin } from "./agents/claude-agent-sdk.js"; export { CodexAgentSdkPlugin } from "./agents/codex-sdk.js"; +export { OpenCodeAgentPlugin } from "./agents/opencode-sdk.js"; export { PiAgentPlugin } from "./agents/pi-sdk.js"; export { AgentRegistry } from "./agents/registry.js"; export { @@ -89,6 +92,7 @@ export function createDefaultAgentRegistry(): AgentRegistry { const registry = new AgentRegistry(); registry.register(new ClaudeAgentSdkPlugin()); registry.register(new CodexAgentSdkPlugin()); + registry.register(new OpenCodeAgentPlugin()); registry.register(new PiAgentPlugin()); // Plugins can contribute additional backends via `agents: []` in their // DeepsecPlugin export. The shape is validated by AgentRegistry at use. @@ -873,15 +877,6 @@ export async function process(params: { // --- Revalidation --- -const SEVERITY_ORDER: Record = { - CRITICAL: 0, - HIGH: 1, - MEDIUM: 2, - HIGH_BUG: 3, - BUG: 4, - LOW: 5, -}; - export async function revalidate(params: { projectId: string; runId?: string; diff --git a/packages/processor/src/triage.ts b/packages/processor/src/triage.ts index 5f07029a..aa670e21 100644 --- a/packages/processor/src/triage.ts +++ b/packages/processor/src/triage.ts @@ -12,10 +12,12 @@ import { writeFileRecord, writeRunMeta, } from "@deepsec/core"; +import { parseAgentJsonArray, writeParseFailureDebug } from "./agents/shared.js"; const TRIAGE_BATCH_SIZE = 30; interface TriageVerdict { + id?: number; title: string; priority: TriagePriority; exploitability: "trivial" | "moderate" | "difficult"; @@ -123,6 +125,7 @@ export async function triage(params: { const findingsList = batch .map((item, idx) => { return `### ${idx + 1}. ${item.finding.title} +- **ID:** ${idx + 1} - **File:** \`${item.record.filePath}\` - **Severity:** ${item.finding.severity} - **Slug:** ${item.finding.vulnSlug} @@ -163,9 +166,13 @@ ${findingsList} ## Output +Return one object per finding above, echoing its **ID** so each verdict can be +matched back to the exact finding it judged (titles are not unique). + \`\`\`json [ { + "id": 1, "title": "exact title", "priority": "P0" | "P1" | "P2" | "skip", "exploitability": "trivial" | "moderate" | "difficult", @@ -193,16 +200,44 @@ ${findingsList} } } - const jsonMatch = resultText.match(/```json\s*([\s\S]*?)```/); - const jsonStr = jsonMatch ? jsonMatch[1].trim() : resultText.trim(); - let verdicts: TriageVerdict[] = []; + // Reuse the project's tolerant parse + fail-loud path (jsonrepair, + // then throw on unrecoverable output). A bare `JSON.parse` swallowed in + // a catch left `verdicts = []`, so a trailing comma or truncated array + // silently dropped the whole batch's triage while still reporting the + // batch as a success — indistinguishable from a clean "nothing to do". + let verdicts: TriageVerdict[]; try { - verdicts = JSON.parse(jsonStr); - } catch {} + verdicts = parseAgentJsonArray({ + resultText, + parseFailurePrefix: "Triage output wasn't a parseable JSON verdict array", + nonArrayMessage: (parsed) => + `Triage output was JSON but not an array of verdicts. Got: ${typeof parsed}`, + }) as TriageVerdict[]; + } catch (parseErr) { + writeParseFailureDebug({ + projectId, + phase: "revalidate", + agentType: "triage", + resultText, + error: parseErr, + batch: batch.map((b) => b.record), + }); + throw parseErr; // surface to the outer catch so the batch is marked failed, not silently empty + } + // Match verdicts back to findings by the 1-based ID emitted in the + // prompt, not by title: titles are free-text and recur across a batch, + // so title-matching mis-attributes verdicts among same-titled findings. + // Fall back to title only when the model omits the id. Guard against a + // single finding being assigned twice (duplicate id/title in the reply). + const assigned = new Set<(typeof batch)[number]>(); for (const verdict of verdicts) { - const item = batch.find((b) => b.finding.title === verdict.title); - if (!item) continue; + const item = + typeof verdict.id === "number" && verdict.id >= 1 && verdict.id <= batch.length + ? batch[verdict.id - 1] + : batch.find((b) => b.finding.title === verdict.title); + if (!item || assigned.has(item)) continue; + assigned.add(item); item.finding.triage = { priority: verdict.priority, diff --git a/packages/scanner/src/__tests__/scan-files.test.ts b/packages/scanner/src/__tests__/scan-files.test.ts index 7f859879..36c2414f 100644 --- a/packages/scanner/src/__tests__/scan-files.test.ts +++ b/packages/scanner/src/__tests__/scan-files.test.ts @@ -1,7 +1,13 @@ import fs from "node:fs"; import os from "node:os"; import path from "node:path"; -import { loadAllFileRecords, readFileRecord, readRunMeta } from "@deepsec/core"; +import { + defineConfig, + loadAllFileRecords, + readFileRecord, + readRunMeta, + setLoadedConfig, +} from "@deepsec/core"; import { afterEach, describe, expect, it } from "vitest"; import { scanFiles } from "../index.js"; @@ -103,4 +109,71 @@ describe("scanFiles()", () => { const after = readFileRecord(projectId, "src/x.ts")!; expect(after.candidates.length).toBe(candCountBefore); }); + + it("throws on unknown matcher slugs", async () => { + const { root, projectId } = makeProject({ "a.ts": "x\n" }); + await expect( + scanFiles({ + projectId, + root, + filePaths: ["a.ts"], + matcherSlugs: ["xss", "does-not-exist", "also-fake"], + }), + ).rejects.toThrow(/Unknown matcher slugs: does-not-exist, also-fake/); + }); +}); + +describe("scanFiles() with config matchers filter", () => { + afterEach(() => { + setLoadedConfig(defineConfig({ projects: [] })); + }); + + it("excludes config-listed matchers from scans", async () => { + const { root, projectId } = makeProject({ + "src/x.ts": 'const q = "SELECT * FROM users WHERE id = " + req.query.id;\n', + }); + setLoadedConfig( + defineConfig({ projects: [], matchers: { exclude: ["js-sql-raw", "dangerous-html"] } }), + ); + + const result = await scanFiles({ projectId, root, filePaths: ["src/x.ts"] }); + expect(result.activeMatchers).not.toContain("js-sql-raw"); + expect(result.skippedMatchers).not.toContain("js-sql-raw"); + }); + + it("restricts scans to matchers listed in config only", async () => { + const { root, projectId } = makeProject({ + "src/x.ts": 'const q = "SELECT * FROM users WHERE id = " + req.query.id;\n', + }); + setLoadedConfig(defineConfig({ projects: [], matchers: { only: ["js-nosql-injection"] } })); + + const result = await scanFiles({ projectId, root, filePaths: ["src/x.ts"] }); + expect(result.activeMatchers).toEqual(["js-nosql-injection"]); + // The sql matcher was excluded by the only-list, not by its gate. + expect([...result.activeMatchers, ...result.skippedMatchers]).not.toContain("js-sql-raw"); + }); + + it("lets explicit matcherSlugs override the config filter", async () => { + const { root, projectId } = makeProject({ + "src/x.ts": 'const q = "SELECT * FROM users WHERE id = " + req.query.id;\n', + }); + setLoadedConfig(defineConfig({ projects: [], matchers: { exclude: ["js-sql-raw"] } })); + + const result = await scanFiles({ + projectId, + root, + filePaths: ["src/x.ts"], + matcherSlugs: ["js-sql-raw"], + }); + expect(result.activeMatchers).toEqual(["js-sql-raw"]); + }); + + it("fails loud on unknown slugs in the config filter", async () => { + const { root, projectId } = makeProject({ "a.ts": "x\n" }); + setLoadedConfig(defineConfig({ projects: [], matchers: { exclude: ["no-such-matcher"] } })); + + await expect(scanFiles({ projectId, root, filePaths: ["a.ts"] })).rejects.toThrow( + /Unknown matcher slug in config matchers: no-such-matcher/, + ); + }); }); diff --git a/packages/scanner/src/index.ts b/packages/scanner/src/index.ts index 32625373..5eb0fe60 100644 --- a/packages/scanner/src/index.ts +++ b/packages/scanner/src/index.ts @@ -7,6 +7,7 @@ import { createRunMeta, dataDir, ensureProject, + getConfig, getDataRoot, getRegistry, projectConfigSchema, @@ -104,6 +105,42 @@ function buildMergedRegistry(): MatcherRegistry { return registry; } +/** + * Resolve the effective matcher slug list for a scan, combining the + * caller's explicit `matcherSlugs` (CLI `--matchers`) with the config's + * `matchers: { only, exclude }` filter (deepsec.config.ts). + * + * Precedence: + * 1. Explicit `matcherSlugs` names the exact set — config `only`/`exclude` + * do not apply. An explicit per-invocation choice beats ambient config. + * 2. Otherwise `only` narrows the registry to the listed slugs. + * 3. `exclude` removes slugs from whatever remains. + * + * Returns `undefined` when nothing filters — callers then use the full + * registry. Unknown slugs in `only`/`exclude` fail loud (same contract as + * `--matchers`) so a typo can't silently widen the run. + */ +function applyConfigMatcherFilter( + registry: MatcherRegistry, + matcherSlugs: string[] | undefined, +): string[] | undefined { + if (matcherSlugs) return matcherSlugs; + const filter = getConfig()?.matchers; + if (!filter || (!filter.only && !filter.exclude)) return undefined; + const selected = filter.only ? [...filter.only] : registry.slugs(); + const excluded = filter.exclude ?? []; + const unknown = registry.unknownSlugs([...selected, ...excluded]); + if (unknown.length > 0) { + throw new Error( + `Unknown matcher slug${unknown.length > 1 ? "s" : ""} in config matchers: ${unknown.join(", ")}.\n` + + ` Available matchers: ${registry.slugs().sort().join(", ")}`, + ); + } + const excludeSet = new Set(excluded); + const effective = selected.filter((slug) => !excludeSet.has(slug)); + return effective; +} + /** Returns the noise tier for a given vulnSlug. Defaults to "normal". */ export function getNoiseTier(slug: string): import("./types.js").NoiseTier { const registry = buildMergedRegistry(); @@ -446,8 +483,23 @@ export async function scan(params: { languageStats: LanguageStats[]; }> { const registry = buildMergedRegistry(); - const allSelected = params.matcherSlugs - ? registry.getBySlugs(params.matcherSlugs) + + // Config `matchers.only/exclude` narrows the set unless the caller named + // the exact matchers explicitly. Unknown slugs in either place fail loud. + const effectiveMatcherSlugs = applyConfigMatcherFilter(registry, params.matcherSlugs); + + if (effectiveMatcherSlugs) { + const unknown = registry.unknownSlugs(effectiveMatcherSlugs); + if (unknown.length > 0) { + throw new Error( + `Unknown matcher slug${unknown.length > 1 ? "s" : ""}: ${unknown.join(", ")}.\n` + + ` Available matchers: ${registry.slugs().sort().join(", ")}`, + ); + } + } + + const allSelected = effectiveMatcherSlugs + ? registry.getBySlugs(effectiveMatcherSlugs) : registry.getAll(); if (allSelected.length === 0) { @@ -631,8 +683,23 @@ export async function scanFiles(params: { skippedMatchers: string[]; }> { const registry = buildMergedRegistry(); - const allSelected = params.matcherSlugs - ? registry.getBySlugs(params.matcherSlugs) + + // Config `matchers.only/exclude` narrows the set unless the caller named + // the exact matchers explicitly. Unknown slugs in either place fail loud. + const effectiveMatcherSlugs = applyConfigMatcherFilter(registry, params.matcherSlugs); + + if (effectiveMatcherSlugs) { + const unknown = registry.unknownSlugs(effectiveMatcherSlugs); + if (unknown.length > 0) { + throw new Error( + `Unknown matcher slug${unknown.length > 1 ? "s" : ""}: ${unknown.join(", ")}.\n` + + ` Available matchers: ${registry.slugs().sort().join(", ")}`, + ); + } + } + + const allSelected = effectiveMatcherSlugs + ? registry.getBySlugs(effectiveMatcherSlugs) : registry.getAll(); if (allSelected.length === 0) { diff --git a/packages/scanner/src/matcher-registry.ts b/packages/scanner/src/matcher-registry.ts index 60350c70..b232b6ed 100644 --- a/packages/scanner/src/matcher-registry.ts +++ b/packages/scanner/src/matcher-registry.ts @@ -21,6 +21,11 @@ export class MatcherRegistry { .filter((m): m is MatcherPlugin => m !== undefined); } + /** Returns slug strings from `requested` that don't exist in the registry. */ + unknownSlugs(requested: string[]): string[] { + return requested.filter((s) => !this.matchers.has(s)); + } + slugs(): string[] { return Array.from(this.matchers.keys()); } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index b72c8c65..900d2752 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -57,6 +57,9 @@ importers: '@openai/codex-sdk': specifier: ^0.156.1 version: 0.156.1 + '@opencode-ai/sdk': + specifier: ^1.18.35 + version: 1.18.35 '@vercel/oidc': specifier: ^3.4.0 version: 3.4.0 @@ -72,12 +75,18 @@ importers: minimatch: specifier: ^10.0.0 version: 10.2.5 + opencode-ai: + specifier: ^1.18.35 + version: 1.18.35 react: specifier: ^19.2.8 version: 19.2.8 tar: specifier: ^7.5.20 version: 7.5.20 + undici: + specifier: ^8.5.0 + version: 8.5.0 devDependencies: '@deepsec/core': specifier: workspace:* @@ -124,9 +133,15 @@ importers: '@openai/codex-sdk': specifier: ^0.156.1 version: 0.156.1 + '@opencode-ai/sdk': + specifier: ^1.18.35 + version: 1.18.35 jsonrepair: specifier: ^3.14.0 version: 3.14.0 + undici: + specifier: ^8.5.0 + version: 8.5.0 packages/scanner: dependencies: @@ -2445,6 +2460,12 @@ packages: dev: false optional: true + /@opencode-ai/sdk@1.18.35: + resolution: {integrity: sha512-cp3IFKwnqgeUbflrrwz/WmG/dALVyM2k0LA1aqCvWe58Tu4utNTNw+BwZzQEML1HxDaLrIB+Fc4Ztb8S16Ijcw==} + dependencies: + cross-spawn: 7.0.6 + dev: false + /@opentelemetry/api@1.9.0: resolution: {integrity: sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg==} engines: {node: '>=8.0.0'} @@ -11133,6 +11154,123 @@ packages: zod: 4.4.3 dev: false + /opencode-ai@1.18.35: + resolution: {integrity: sha512-tDQKThIZ3amp8NSn3XDP2Dj5BlhRbZC0VkIxgYuf6gvRSXkseqD31j/U4KfpPwha593i8M59hdsDfz32yJx4Og==} + cpu: [arm64, x64] + os: [darwin, linux, win32] + hasBin: true + requiresBuild: true + optionalDependencies: + opencode-darwin-arm64: 1.18.35 + opencode-darwin-x64: 1.18.35 + opencode-darwin-x64-baseline: 1.18.35 + opencode-linux-arm64: 1.18.35 + opencode-linux-arm64-musl: 1.18.35 + opencode-linux-x64: 1.18.35 + opencode-linux-x64-baseline: 1.18.35 + opencode-linux-x64-baseline-musl: 1.18.35 + opencode-linux-x64-musl: 1.18.35 + opencode-windows-arm64: 1.18.35 + opencode-windows-x64: 1.18.35 + opencode-windows-x64-baseline: 1.18.35 + dev: false + + /opencode-darwin-arm64@1.18.35: + resolution: {integrity: sha512-WlM5yNbT6OppyXBaTgnri0tnsi/axeLl16vlIrfIdjm4yEvuZVD0JBpBNOLBI21jig0Rw7/HRAO0XWokgJge9g==} + cpu: [arm64] + os: [darwin] + requiresBuild: true + dev: false + optional: true + + /opencode-darwin-x64-baseline@1.18.35: + resolution: {integrity: sha512-Fk6bhyYZ2M8H10hx0/5FUkomLyl4tYN1pJl4Y6tAy8vnO4rSve90QPup6ngXPIYvCbB2JbaDKsgl2Q8cJnBxeA==} + cpu: [x64] + os: [darwin] + requiresBuild: true + dev: false + optional: true + + /opencode-darwin-x64@1.18.35: + resolution: {integrity: sha512-beywtZN8juHfH3BzJ79/v0MigR5XszjXkIH8IG+uMFRKdqzYHpZN7Wm5uTj3zTxnx4D95XYSV6ICkiTsU829zQ==} + cpu: [x64] + os: [darwin] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-arm64-musl@1.18.35: + resolution: {integrity: sha512-f3uFZXwXPYHx8frgDzBTFFQmzVjm50Yg5gOSScPoXTgURZRfeU2zgV5iCvBhUSjlA0wvpZtBswUqwN9Kd4n8bw==} + cpu: [arm64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-arm64@1.18.35: + resolution: {integrity: sha512-5IgrvyoxQc3altzqsGPM5Aigf/uzC/pMFG2Kt4w8vJEtJKKem4Hl9wEVmsZxMYFZDzyx7e0MuBkOAzt6AD++1A==} + cpu: [arm64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-x64-baseline-musl@1.18.35: + resolution: {integrity: sha512-JtiM7KiF7Tf/aZKfPYy0dOom2Ty1ZxnFp1mUJzUuNXQHeCJAwLcaxC39OxtRbu7cCTmIV2pI5m+bVizxW1vE8w==} + cpu: [x64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-x64-baseline@1.18.35: + resolution: {integrity: sha512-M4SO/A3i2pfOxwBl2C4w6ZgjUx1CH0Y1+yIrc2OdagH3ZOYuhCiFR2FVxhOwMxA/CxlMrS3Z6xxY8WSZCGfuYw==} + cpu: [x64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-x64-musl@1.18.35: + resolution: {integrity: sha512-RN7vMSmb/Z0ibxRxZXW3oi7MhNTarS/jHXG2YwHkpC1DSDfur5f/K6UjajRdtgiBEv3W8A7bEej+2cRvXAG95w==} + cpu: [x64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-linux-x64@1.18.35: + resolution: {integrity: sha512-xs/DPt8bK6NTWHpOlVJDWC/P3BnAuAQFHwa+mjrbxhDGd0Pi/GGWGjh4Sr4cpv6KwvfUKVoLKfHc/J8Nnz0XMg==} + cpu: [x64] + os: [linux] + requiresBuild: true + dev: false + optional: true + + /opencode-windows-arm64@1.18.35: + resolution: {integrity: sha512-zPTMxYzUP+7rsiZWE2ejPOJLgp4+IyUUXvr5vh/ufx1v5Pq05sPy4hvS1uZJFS/KxFQ09MAiEJ2LKUJ7CHCGig==} + cpu: [arm64] + os: [win32] + requiresBuild: true + dev: false + optional: true + + /opencode-windows-x64-baseline@1.18.35: + resolution: {integrity: sha512-L4gD+osKK5YRY1ZDdH87dzvG5MAUpXci4CgBrJ5+GCxJnrBa+7sJqpgSHZPQBNRRPuT0nPqM3uasfFPEzkLSmQ==} + cpu: [x64] + os: [win32] + requiresBuild: true + dev: false + optional: true + + /opencode-windows-x64@1.18.35: + resolution: {integrity: sha512-KFFt0MPMoKsniQ9NO0HqVIYM7ZoVwKNT+xg6oNOk0qRgPk2sAHCDcKZCnAuzVmdEzG7MJfDdkav9kjVCZce6sQ==} + cpu: [x64] + os: [win32] + requiresBuild: true + dev: false + optional: true + /optionator@0.9.4: resolution: {integrity: sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g==} engines: {node: '>= 0.8.0'}