Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions docs/api-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,7 @@ Exported from `@loushy/build-ai-agent/testing` (see [Testing agents](testing.md)
| `ToolInput<typeof t>`, `ToolOutput<typeof t>` | Argument and result types of a defined tool. |
| `ToolRegistry` | Holds the tools an agent config refers to (advanced: `register(tool)` or `register(name, descriptor)`). |
| `httpTool`, `currentDateTool`, `dayNameTool` | Built-in tools. |
| `createTodoTools({ store?, onChange? })` | `todo_write` / `todo_read` tools (plus `getTodos()`) so agents can plan and track multi-step work; see [Todo tools](#todo-tools). |
| `loadMcpTools(client, connectionName)` | Load a connected MCP server's tools as `ToolDescriptor`s. Available from the package root, `@loushy/build-ai-agent/tools`, and `@loushy/build-ai-agent/mcp`. |

```ts
Expand Down Expand Up @@ -232,6 +233,31 @@ non-text parts the result is `{ structuredContent, text, content }` with
`content` holding only the non-text parts, so images, audio and resources are
never dropped.

### Todo tools

`createTodoTools(options?)` gives long-running agents a plan to track:
`todo_write` replaces the whole list (`{ id?, content, status }` items, status
`pending` | `in_progress` | `completed`, at most one `in_progress`) and returns
the list plus counts; `todo_read` returns it. Ids are assigned automatically and
stay stable when a later write repeats an item's content. Invalid lists (for
example two `in_progress`) reach the model as a structured tool error so it can
retry. The list lives in memory per call; pass `store` (`{ get, set }`) to
persist it, and `onChange` to update a UI.

```ts
import { createAgent, createTodoTools, type TodoStore } from '@loushy/build-ai-agent';

const todos = createTodoTools({ onChange: (list) => console.log(list.length, 'todos') });
const planner = createAgent({ prompt: 'Plan multi-step work, then do it.', provider, tools: todos.tools });

await planner.send('Migrate the repo to ESM');
console.log(await todos.getTodos()); // [{ id: 'todo_1', content: '...', status: 'completed' }, ...]

// Persist the list somewhere else:
let saved: Awaited<ReturnType<TodoStore['get']>> = [];
createTodoTools({ store: { get: () => saved, set: (next) => void (saved = next) } });
```

### Tool errors

A tool call can fail in two ways. In both, the run continues and the model
Expand Down Expand Up @@ -442,6 +468,54 @@ failed - and the reason (never a secret or signature) is logged at `warn`
level. Serve webhooks over HTTPS (terminate TLS in front of the adapter) so
tokens and payloads are not sent in clear text.

### Slack request signatures

`SlackTriggerAdapter.handleRequest({ headers, rawBody })` handles a raw Slack
Events API request and returns the `{ status, body }` to send back. **Set
`signingSecret` for any endpoint reachable from outside your machine**: without
it, anyone who can reach the endpoint can run your agent, and `listen()` logs a
one-time warning through `options.logger`. With it, every request is verified
as [Slack documents](https://docs.slack.dev/authentication/verifying-requests-from-slack)
before the body is parsed: HMAC-SHA256 over `v0:{X-Slack-Request-Timestamp}:{raw body}`,
compared in constant time with `X-Slack-Signature` (`v0=<hex>`), and requests
more than five minutes old are rejected. Failures get a generic
`401 {"error":"Unauthorized"}`; the reason (never a secret or signature) is
logged at `warn` level. The signed `url_verification` handshake is answered
after verification.

```ts
import { createAgent, createMockProvider } from '@loushy/build-ai-agent';
import { SlackTriggerAdapter, verifySlackSignature } from '@loushy/build-ai-agent/triggers';
import * as http from 'node:http';

const agent = createAgent({ prompt: 'You are helpful.', provider: createMockProvider() });
const slack = new SlackTriggerAdapter({ signingSecret: process.env.SLACK_SIGNING_SECRET });
slack.listen(agent, (input) => agent.send(input));

http
.createServer((req, res) => {
const chunks: Buffer[] = [];
req.on('data', (chunk: Buffer) => chunks.push(chunk));
req.on('end', async () => {
const { status, body } = await slack.handleRequest({ headers: req.headers, rawBody: Buffer.concat(chunks) });
res.writeHead(status, { 'Content-Type': 'application/json' }).end(JSON.stringify(body));
});
})
.listen(3000);

// Your own handler (slash commands, interactivity)? Verify the RAW body yourself:
const authentic = verifySlackSignature({
signingSecret: process.env.SLACK_SIGNING_SECRET ?? '',
timestamp: '1700000000', // the X-Slack-Request-Timestamp header
signature: 'v0=...', // the X-Slack-Signature header
rawBody: '{"type":"url_verification"}',
});
```

`handleRequest` answers Slack after the agent finishes; Slack expects a reply
within three seconds, so for slow agents acknowledge first and run the agent in
the background.

### Cron schedules

`CronTriggerAdapter` takes either a fixed `intervalMs` or a real cron
Expand Down
4 changes: 4 additions & 0 deletions examples/ops-pipeline/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ required for the default demo:

- `SLACK_WEBHOOK_URL` - real Slack Incoming Webhook (see
`src/tools/built-in/slack.ts`)
- `SLACK_SIGNING_SECRET` - your Slack app's signing secret. When set,
`POST /slack/interactions` verifies every request's `X-Slack-Signature`
(via the SDK's `verifySlackSignature`) and answers 401 otherwise. Set it
whenever the endpoint is reachable from the internet.
- `OPENAI_API_KEY` / `ANTHROPIC_API_KEY` - real LLM provider (pass a real
`LLMProvider` to `startOpsPipeline({ provider })` instead of the default
`demoProvider`)
Expand Down
29 changes: 29 additions & 0 deletions examples/ops-pipeline/index.test.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { createHmac } from 'node:crypto';
import { describe, it, expect, afterEach } from 'vitest';
import { startOpsPipeline, OpsPipelineHandle } from './index';
import { createDemoProvider } from './demoProvider';
Expand Down Expand Up @@ -68,3 +69,31 @@ describe('ops-pipeline end-to-end (LOU-J8, against mocks - zero external network
expect(github.createdPullRequests[0].body).toContain('OrderService.java');
});
});

describe('ops-pipeline Slack signature verification (LOU-D18)', () => {
let handle: OpsPipelineHandle | undefined;

afterEach(async () => {
await handle?.close();
handle = undefined;
});

it('rejects unsigned interaction callbacks and accepts correctly signed ones when a signing secret is set', async () => {
const secret = 'ops-signing-secret';
handle = await startOpsPipeline({ slackSigningSecret: secret, monitorPort: 0, slackPort: 0 });
const url = `http://127.0.0.1:${handle.slack.port}/slack/interactions`;
const body = JSON.stringify({ type: 'block_actions', actions: [{ action_id: 'other', value: 'x' }] });

const unsigned = await fetch(url, { method: 'POST', body });
expect(unsigned.status).toBe(401);

const timestamp = String(Math.floor(Date.now() / 1000));
const signature = `v0=${createHmac('sha256', secret).update(`v0:${timestamp}:${body}`).digest('hex')}`;
const signed = await fetch(url, {
method: 'POST',
headers: { 'x-slack-request-timestamp': timestamp, 'x-slack-signature': signature },
body,
});
expect(signed.status).toBe(200);
});
});
24 changes: 24 additions & 0 deletions examples/ops-pipeline/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import * as http from 'node:http';
import { AgentType, ToolDescriptor } from '../../src/types';
import { ToolRegistry } from '../../src/tools';
import { verifySlackSignature } from '../../src/triggers';
import { LLMProvider } from '../../src/providers/llm';
import { startMonitorServer, MonitorServerHandle, StartMonitorServerOptions } from './monitor';
import { createFixerDelegateTool, buildFixerAgent } from './fixer';
Expand Down Expand Up @@ -84,6 +85,11 @@ export interface OpsPipelineDeps {
slackHost?: string;
slackPort?: number;
channel?: string;
/**
* Slack signing secret. When set (default: `SLACK_SIGNING_SECRET`), requests to
* POST /slack/interactions must carry a valid Slack signature or get a 401.
*/
slackSigningSecret?: string;
/**
* Guardrails to run before considering the GitHub PR call. Defaults to
* handleFixerPatch()'s own defaults (real secretScanGuardrail + a
Expand Down Expand Up @@ -135,9 +141,22 @@ interface SlackRouteContext {
githubCreatePrTool: ToolDescriptor;
slackTool: ToolDescriptor;
channel: string;
signingSecret?: string;
guardrails?: Guardrail[];
}

/** True when no signing secret is configured, or the request carries a valid Slack signature over the raw body. */
function isAuthenticSlackRequest(req: http.IncomingMessage, raw: string, signingSecret: string | undefined): boolean {
if (!signingSecret) return true;
const header = (name: string) => (typeof req.headers[name] === 'string' ? (req.headers[name] as string) : undefined);
return verifySlackSignature({
signingSecret,
timestamp: header('x-slack-request-timestamp'),
signature: header('x-slack-signature'),
rawBody: raw,
});
}

/**
* Handles one POST /slack/interactions request: resumes the paused run, then -
* if the fixer produced a patch - sends it through the guardrail-gated PR path.
Expand All @@ -149,6 +168,10 @@ async function handleSlackInteractionRequest(
): Promise<void> {
try {
const raw = await readBody(req);
if (!isAuthenticSlackRequest(req, raw, ctx.signingSecret)) {
sendJson(res, 401, { error: 'Unauthorized' });
return;
}
const payload = parseSlackInteractionBody(raw);
const approvalId = extractApprovalIdFromInteraction(payload);

Expand Down Expand Up @@ -254,6 +277,7 @@ export async function startOpsPipeline(deps: OpsPipelineDeps = {}): Promise<OpsP
githubCreatePrTool,
slackTool,
channel,
signingSecret: deps.slackSigningSecret ?? process.env.SLACK_SIGNING_SECRET,
guardrails: deps.guardrails,
},
deps.slackPort ?? 0,
Expand Down
1 change: 1 addition & 0 deletions src/tools/built-in/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ export * from './email';
export * from './jira';
export * from './github';
export * from './slack';
export * from './todo';
155 changes: 155 additions & 0 deletions src/tools/built-in/todo.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
import { describe, it, expect, vi } from 'vitest';
import { createTodoTools, type Todo, type TodoStore } from './todo';
import { createAgent } from '../../createAgent';
import { mockModel } from '../../testing';

/** The JSON the model received as the result of tool call `n` (0-based), read from the last recorded request. */
function toolResults(model: ReturnType<typeof mockModel>): Array<Record<string, unknown>> {
const last = model.calls[model.calls.length - 1];
return last.messages.filter((m) => m.role === 'tool').map((m) => JSON.parse(String(m.content)));
}

describe('createTodoTools', () => {
it('lets a run write, update and read the list, keeping ids stable', async () => {
const todos = createTodoTools();
const model = mockModel([
{
toolCalls: [
{
name: 'todo_write',
args: {
todos: [
{ content: 'Write tests', status: 'in_progress' },
{ content: 'Ship it', status: 'pending' },
],
},
},
],
},
{
toolCalls: [
{
name: 'todo_write',
args: {
todos: [
{ content: 'Write tests', status: 'completed' },
{ content: 'Ship it', status: 'in_progress' },
{ content: 'Celebrate', status: 'pending' },
],
},
},
{ name: 'todo_read' },
],
},
'All done.',
]);
const agent = createAgent({ prompt: 'Plan then act.', provider: model, tools: todos.tools });

const result = await agent.send('Do the thing');

expect(result.text).toBe('All done.');
const [, , read] = toolResults(model);
const final = await todos.getTodos();
expect(final.map((t) => [t.id, t.content, t.status])).toEqual([
['todo_1', 'Write tests', 'completed'],
['todo_2', 'Ship it', 'in_progress'],
['todo_3', 'Celebrate', 'pending'],
]);
expect(read).toEqual({
todos: final,
counts: { pending: 1, in_progress: 1, completed: 1, total: 3 },
});
});

it('assigns ids once, keeps explicit ids and avoids collisions', async () => {
const { todoWrite, getTodos } = createTodoTools();
const exec = todoWrite.tool.execute as (args: unknown, ctx: unknown) => Promise<unknown>;
await exec({ todos: [{ id: 'todo_1', content: 'a', status: 'pending' }, { content: 'b', status: 'pending' }] }, {});
expect((await getTodos()).map((t) => t.id)).toEqual(['todo_1', 'todo_2']);
await exec({ todos: [{ content: 'b', status: 'completed' }, { content: 'c', status: 'pending' }] }, {});
expect((await getTodos()).map((t) => t.id)).toEqual(['todo_2', 'todo_3']);
});

it('returns validation failures (two in_progress, duplicate ids, empty content) to the model as structured errors', async () => {
const todos = createTodoTools();
const model = mockModel([
{
toolCalls: [
{
name: 'todo_write',
args: { todos: [{ content: 'a', status: 'in_progress' }, { content: 'b', status: 'in_progress' }] },
},
{ name: 'todo_write', args: { todos: [{ id: 'x', content: 'a', status: 'pending' }, { id: 'x', content: 'b', status: 'pending' }] } },
{ name: 'todo_write', args: { todos: [{ content: ' ', status: 'pending' }] } },
],
},
'gave up',
]);
const agent = createAgent({ prompt: 'p', provider: model, tools: todos.tools });

await agent.send('go');

const [two, dup, empty] = toolResults(model);
expect(two).toMatchObject({ error: 'ToolArgumentsValidationError', toolName: 'todo_write' });
expect(String(two.message)).toMatch(/at most one todo may be in_progress/i);
expect(String(dup.message)).toMatch(/unique/);
expect(empty).toMatchObject({ error: 'ToolArgumentsValidationError' });
expect(await todos.getTodos()).toEqual([]);
});

it('calls onChange with the new list after each write', async () => {
const onChange = vi.fn();
const todos = createTodoTools({ onChange });
const model = mockModel([
{ toolCalls: [{ name: 'todo_write', args: { todos: [{ content: 'a', status: 'pending' }] } }] },
{ toolCalls: [{ name: 'todo_read' }] },
'ok',
]);

await createAgent({ prompt: 'p', provider: model, tools: todos.tools }).send('go');

expect(onChange).toHaveBeenCalledTimes(1);
expect(onChange).toHaveBeenCalledWith([{ id: 'todo_1', content: 'a', status: 'pending' }]);
});

it('reads and writes through a custom (async) store', async () => {
let saved: Todo[] = [{ id: 'todo_7', content: 'restored', status: 'pending' }];
const store: TodoStore = {
get: async () => saved,
set: async (next) => {
saved = next;
},
};
const todos = createTodoTools({ store });
const model = mockModel([
{ toolCalls: [{ name: 'todo_read' }] },
{ toolCalls: [{ name: 'todo_write', args: { todos: [{ content: 'restored', status: 'completed' }, { content: 'new', status: 'pending' }] } }] },
'ok',
]);

await createAgent({ prompt: 'p', provider: model, tools: todos.tools }).send('go');

expect(saved).toEqual([
{ id: 'todo_7', content: 'restored', status: 'completed' },
{ id: 'todo_8', content: 'new', status: 'pending' },
]);
expect(await todos.getTodos()).toBe(saved);
});

it('keeps separate in-memory lists per createTodoTools() call', async () => {
const a = createTodoTools();
const b = createTodoTools();
await (a.todoWrite.tool.execute as (args: unknown, ctx: unknown) => Promise<unknown>)(
{ todos: [{ content: 'only a', status: 'pending' }] },
{}
);
expect(await a.getTodos()).toHaveLength(1);
expect(await b.getTodos()).toHaveLength(0);
});

it('describes when to use each tool', () => {
const { todoWrite, todoRead } = createTodoTools();
expect(todoWrite.description).toMatch(/multi-step|three or more steps/);
expect(todoRead.name).toBe('todo_read');
});
});
Loading
Loading