LOU-J: Cross-harness spec & flagship pipeline

examples/README.md

@@ -9,6 +9,12 @@ to try.
 
 A question-answering agent scoped to a single fixed document (LOU-H10).
 
+## [ops-pipeline](./ops-pipeline)
+
+A flagship end-to-end pipeline: a Grafana/Datadog monitor delegates a fix (behind a real
+human approval gate, with a Slack "Fix it" button) to a fixer agent, whose patch is
+guardrail-gated before a GitHub PR is opened (LOU-J4-J9).
+
 ## [research-assistant](./research-assistant)
 
 A research agent with the built-in `http` tool wired up (LOU-H10).

examples/ops-pipeline/.env.example

@@ -0,0 +1,17 @@
+# All of these are OPTIONAL - the default `npm run pipeline:demo` runs
+# entirely against mocks/ and needs none of them set.
+
+# Real Slack Incoming Webhook URL (src/tools/built-in/slack.ts).
+SLACK_WEBHOOK_URL=
+
+# Real LLM provider credentials, only used if you wire a real LLMProvider
+# into startOpsPipeline({ provider }) instead of the default demoProvider.
+OPENAI_API_KEY=
+ANTHROPIC_API_KEY=
+
+# Real GitHub token, owner/repo, only used if you wire a real GitHubTools
+# instance into startOpsPipeline({ githubCreatePrTool }) instead of the
+# default mock GitHub tool.
+GITHUB_TOKEN=
+GITHUB_OWNER=
+GITHUB_REPO=

examples/ops-pipeline/README.md

@@ -0,0 +1,97 @@
+# ops-pipeline (LOU-J8)
+
+An end-to-end "flagship pipeline" demo composing every LOU-J primitive:
+
+1. **Monitor** (LOU-J4) - a `POST /webhook` listener that receives a Grafana/
+   Datadog-shaped `ErrorSignal`, dedupes it by signature, and asks the
+   monitor agent to triage it.
+2. **Approval gate** (LOU-C, via the real `ApprovalGate`/`resumeAfterApproval`)
+   - the monitor's delegation to the fixer agent is flagged
+   `needsApproval: true`, so it always pauses for a human before the fixer
+   agent ever runs.
+3. **Slack "Fix it" button** (LOU-J5) - the pause posts a Slack alert with a
+   "Fix it" button; clicking it (`POST /slack/interactions`) resumes the
+   paused run through the REAL `resumeAfterApproval()`.
+4. **Fixer** (LOU-J6) - once approved, the fixer agent diagnoses the error
+   and produces a unified diff.
+5. **Guardrail-gated PR** (LOU-J7) - the diff is checked against the REAL
+   `secretScanGuardrail` + a diff-size cap BEFORE a GitHub PR is even
+   considered; a failure notifies Slack instead of ever calling GitHub.
+
+By default this runs entirely against the `mocks/` implementations (mock
+Slack API, mock GitHub API, and a deterministic scripted LLM provider), so
+it needs **zero external network access** and zero API keys.
+
+## Setup
+
+1. Install dependencies (from the repo root):
+
+   ```
+   npm install
+   ```
+
+2. Copy the example env file (only needed if you want to point this at
+   REAL Slack/GitHub/LLM providers instead of the mocks - the demo works
+   with none of this set):
+
+   ```
+   cp examples/ops-pipeline/.env.example .env
+   ```
+
+3. Start the demo pipeline:
+
+   ```
+   npm run pipeline:demo
+   ```
+
+   This starts two local listeners (both bound to `127.0.0.1`):
+   - `http://127.0.0.1:8787/webhook` - the monitor's error-signal intake
+   - `http://127.0.0.1:8788/slack/interactions` - the Slack button callback
+
+4. In a second terminal, POST a synthetic error at the running pipeline:
+
+   ```
+   npm run pipeline:demo:trigger
+   ```
+
+   Watch the first terminal's output - this triggers the monitor agent,
+   which pauses for approval and posts a (mock) Slack alert with a "Fix
+   it" button.
+
+5. Simulate clicking "Fix it" (in the real integration this is a real
+   Slack button click; here it's a plain POST simulating Slack's
+   interaction callback - substitute the `approvalId` your mock Slack
+   tool logged):
+
+   ```
+   curl -X POST http://127.0.0.1:8788/slack/interactions \
+     -H 'Content-Type: application/json' \
+     -d '{"type":"block_actions","actions":[{"action_id":"fix_it","value":"<approvalId>"}]}'
+   ```
+
+   This resumes the paused run, runs the fixer agent, and - if the
+   guardrail check passes - creates a (mock) GitHub pull request.
+
+## Files
+
+- `monitor.ts` - LOU-J4: ErrorSignal type, dedup, `POST /webhook`.
+- `fixer.ts` - LOU-J6: fixer agent, diff extraction, delegate-tool wiring.
+- `guardedPr.ts` - LOU-J7: guardrail-gated PR creation.
+- `demoProvider.ts` - a deterministic, zero-network scripted LLM provider
+  used by the demo in place of a real model.
+- `mocks/` - mock Grafana sender, mock Slack API, mock GitHub API (same
+  call shapes as the real tools).
+- `index.ts` - wires everything into one process (`npm run pipeline:demo`).
+
+## Real credentials (optional)
+
+Set these to run against real services instead of the mocks - none are
+required for the default demo:
+
+- `SLACK_WEBHOOK_URL` - real Slack Incoming Webhook (see
+  `src/tools/built-in/slack.ts`)
+- `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` - real LLM provider (pass a real
+  `LLMProvider` to `startOpsPipeline({ provider })` instead of the default
+  `demoProvider`)
+- A real GitHub token (`src/tools/built-in/github.ts`'s `createGitHubTools`)
+  passed as `githubCreatePrTool` to `startOpsPipeline()`

examples/ops-pipeline/demoProvider.ts

@@ -0,0 +1,106 @@
+/**
+ * Deterministic, zero-network scripted LLMProvider for the ops-pipeline
+ * demo (LOU-J8). Not a "real" provider - it inspects the conversation
+ * shape to decide what a real model would plausibly do at each step:
+ *
+ *  - Monitor agent turn (tools present, no tool result yet in the
+ *    conversation): calls the `delegate_to_fixer` tool with the
+ *    triggering user message as the delegated task.
+ *  - Monitor agent's follow-up turn (tools present, a tool result already
+ *    in the conversation): finishes with a short confirmation, no further
+ *    tool calls.
+ *  - Fixer agent turn (no tools - the fixer agent is never given a tool
+ *    registry): returns a fenced ```diff block diagnosing the demo's
+ *    synthetic NullPointerException.
+ */
+import { GenerateOptions, GenerateResult, LLMProvider, StreamResult } from '../../src/providers';
+
+export const DEMO_FIXER_DIFF_RESPONSE = [
+  'Root cause: `order` is not null-checked before `charge()` is called.',
+  '',
+  '```diff',
+  '--- a/src/OrderService.java',
+  '+++ b/src/OrderService.java',
+  '@@ -39,7 +39,9 @@',
+  '   public void checkout(Order order) {',
+  '-    charge(order.getPayment());',
+  '+    if (order.getPayment() == null) {',
+  '+      throw new IllegalArgumentException("payment is required");',
+  '+    }',
+  '+    charge(order.getPayment());',
+  '   }',
+  '```',
+].join('\n');
+
+function usageFor(text: string): GenerateResult['usage'] {
+  const tokens = Math.max(1, Math.ceil(text.length / 4));
+  return { promptTokens: tokens, completionTokens: tokens, totalTokens: tokens * 2 };
+}
+
+export function createDemoProvider(): LLMProvider {
+  const generate = async (options: GenerateOptions): Promise<GenerateResult> => {
+    const hasTools = !!options.tools && options.tools.length > 0;
+    const alreadyDelegated = options.messages.some((m) => m.role === 'tool');
+
+    if (hasTools && !alreadyDelegated) {
+      const lastUser = [...options.messages].reverse().find((m) => m.role === 'user');
+      return {
+        text: '',
+        finishReason: 'tool_calls',
+        usage: usageFor(lastUser?.content ?? ''),
+        toolCalls: [
+          {
+            id: 'call_delegate_1',
+            type: 'function',
+            function: {
+              name: 'delegate_to_fixer',
+              arguments: JSON.stringify({ task: lastUser?.content ?? '' }),
+            },
+          },
+        ],
+      };
+    }
+
+    if (hasTools && alreadyDelegated) {
+      const text = 'Delegated to the fixer agent for review.';
+      return { text, finishReason: 'stop', usage: usageFor(text) };
+    }
+
+    // Fixer agent turn: no tools registered for it.
+    return {
+      text: DEMO_FIXER_DIFF_RESPONSE,
+      finishReason: 'stop',
+      usage: usageFor(DEMO_FIXER_DIFF_RESPONSE),
+    };
+  };
+
+  return {
+    name: 'ops-pipeline-demo',
+    generate,
+    stream: async (options: GenerateOptions): Promise<StreamResult> => {
+      const result = await generate(options);
+      async function* full() {
+        yield { type: 'text-delta' as const, textDelta: result.text };
+        yield {
+          type: 'finish' as const,
+          finishReason: result.finishReason,
+          usage: result.usage,
+        };
+      }
+      async function* textOnly() {
+        yield result.text;
+      }
+      return {
+        fullStream: full(),
+        textStream: textOnly(),
+        text: Promise.resolve(result.text),
+        usage: Promise.resolve(result.usage),
+        finishReason: Promise.resolve(result.finishReason),
+        toolCalls: Promise.resolve(result.toolCalls ?? []),
+      };
+    },
+    supportsTools: () => true,
+    supportsStreaming: () => true,
+    getModels: async () => ['ops-pipeline-demo'],
+  };
+}

examples/ops-pipeline/fixer.test.ts

@@ -0,0 +1,102 @@
+import { describe, it, expect, vi } from 'vitest';
+import { runFixer, extractDiffBlock, EmptyPatchError, FixRequest, createFixerDelegateTool } from './fixer';
+import { LLMProvider, GenerateResult } from '../../src/providers';
+
+function makeScriptedProvider(text: string): LLMProvider {
+  const result: GenerateResult = {
+    text,
+    finishReason: 'stop',
+    usage: { promptTokens: 10, completionTokens: 10, totalTokens: 20 },
+  };
+  return {
+    name: 'mock',
+    generate: vi.fn().mockResolvedValue(result),
+    stream: vi.fn() as any,
+    supportsTools: () => true,
+    supportsStreaming: () => true,
+    getModels: async () => [],
+  };
+}
+
+const request: FixRequest = {
+  errorSignature: 'sig-1',
+  logs: 'NullPointerException at OrderService.java:42',
+  files: ['src/OrderService.java'],
+};
+
+const FENCED_DIFF_RESPONSE = [
+  'Here is the fix:',
+  '',
+  '```diff',
+  '--- a/src/OrderService.java',
+  '+++ b/src/OrderService.java',
+  '@@ -40,3 +40,3 @@',
+  '-  charge(null);',
+  '+  charge(order);',
+  '```',
+].join('\n');
+
+describe('extractDiffBlock', () => {
+  it('extracts a fenced ```diff block', () => {
+    const patch = extractDiffBlock(FENCED_DIFF_RESPONSE);
+    expect(patch).toContain('--- a/src/OrderService.java');
+    expect(patch).toContain('+  charge(order);');
+  });
+
+  it('extracts a diff-looking plain fenced block', () => {
+    const text = ['```', '--- a/foo.txt', '+++ b/foo.txt', '@@ -1 +1 @@', '-old', '+new', '```'].join(
+      '\n'
+    );
+    expect(extractDiffBlock(text)).toContain('+new');
+  });
+
+  it('falls back to scanning for unified-diff markers with no fencing', () => {
+    const text = ['I fixed it.', '--- a/foo.txt', '+++ b/foo.txt', '@@ -1 +1 @@', '-old', '+new'].join(
+      '\n'
+    );
+    expect(extractDiffBlock(text)).toContain('+new');
+  });
+
+  it('returns an empty string when nothing diff-shaped is present', () => {
+    expect(extractDiffBlock('I could not find a fix.')).toBe('');
+  });
+});
+
+describe('runFixer', () => {
+  it('with a deterministic mock provider scripted to return a fenced diff, returns a non-empty patch referencing the expected file', async () => {
+    const provider = makeScriptedProvider(FENCED_DIFF_RESPONSE);
+
+    const { patch } = await runFixer(request, provider);
+
+    expect(patch.length).toBeGreaterThan(0);
+    expect(patch).toContain('src/OrderService.java');
+  });
+
+  it('throws EmptyPatchError when the fixer response has no extractable diff', async () => {
+    const provider = makeScriptedProvider('I am not sure how to fix this.');
+
+    await expect(runFixer(request, provider)).rejects.toBeInstanceOf(EmptyPatchError);
+  });
+});
+
+describe('createFixerDelegateTool', () => {
+  it('delegates to the fixer agent and extracts a patch from its response', async () => {
+    const provider = makeScriptedProvider(FENCED_DIFF_RESPONSE);
+    const fixerAgent = { name: 'fixer', agentType: 0 as any, prompt: 'fix it' };
+
+    const delegateTool = createFixerDelegateTool({ agent: fixerAgent, provider });
+    const result: any = await delegateTool.tool.execute!({ task: 'fix sig-1' }, {} as any);
+
+    expect(result.patch).toContain('src/OrderService.java');
+  });
+
+  it('throws EmptyPatchError when the delegated fixer agent has no extractable diff', async () => {
+    const provider = makeScriptedProvider('I am not sure how to fix this.');
+    const fixerAgent = { name: 'fixer', agentType: 0 as any, prompt: 'fix it' };
+
+    const delegateTool = createFixerDelegateTool({ agent: fixerAgent, provider });
+    await expect(delegateTool.tool.execute!({ task: 'fix sig-1' }, {} as any)).rejects.toBeInstanceOf(
+      EmptyPatchError
+    );
+  });
+});