Skip to content
10 changes: 8 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,16 @@ opt-out.

- The `lousho observe` daemon: a local collector (loopback, `http://127.0.0.1:7777`, SQLite at `~/.lousho/observe.db`) that gathers every agent's traces, heartbeats and eval scores when `LOUSHO_OBSERVE` is set — no per-agent code changes. `lousho observe start|status|stop|rebuild-rollups|token create`; a second `start` on a busy port attaches when it is already the observe daemon. See [Observability](docs/observability.md#the-observe-daemon).
- Agent Forge gains Observe tabs — Traces, Insights, Monitoring, Scores — reading the daemon through a token-gated `/observe/**` proxy on the studio server.
- Prompt registry: `createAgent({ prompt: 'name@version' })` or `LOUSHO_PROMPT` resolves versioned instructions from the daemon at build (embedded `instructions` are the fallback); every run records the version used. `lousho prompt push|list|get` manages it.
- Prompt registry: `createAgent({ prompt: 'name@version' })` or `LOUSHO_PROMPT` resolves versioned instructions from the daemon at build; every run records the version used. `lousho prompt push|list|get` manages it. When the daemon is unreachable, embedded `instructions` are used; with no `instructions` the agent fails closed: every run rejects with `LOUSHO_PROMPT_UNRESOLVED` instead of running without a system prompt. A failed fetch is not cached for good: runs in the next 30 seconds resolve at once, then the next run fetches again (waiting up to 2.5 seconds).
- Context hub: `hub()` / `hubTools()` publish and query namespaced shared memory scoped `private` (the writing agent) or `fleet` (every agent on the daemon).
- Fleet monitoring: heartbeat-derived agent status, `agent.deregister` on graceful exit, a crashed agent's open sessions and unfinished observations flip to `interrupted`, and webhook alerts (`--webhook`, `alerts.json`) on down transitions and error-rate thresholds.
- Eval scores report to the daemon: every `AssertionResult` lands as a score row on the exercised run's trace (`source: eval` or `judge`); `lousho eval --observe` forces emission.
- Eval scores report to the daemon: every `AssertionResult` lands as a score row on the exercised run's trace (`source: eval` or `judge`); `lousho eval --observe` forces emission. Each case's `eval` span and score rows are flushed when the case finishes, each wait capped at 1 second; the scores share the reporter's queue, so they go out after the span and are retried while the daemon is down. A wait that runs out skips the waits of the next 30 seconds of cases.
- Reporting to the daemon: spans that finished without an error report status `ok`. A down daemon's batches are requeued and retried with exponential backoff (1 s up to 30 s); batches are capped at 1000 events and 4 MB, and a batch the daemon refuses (`400`/`413`) is dropped and counted in `dropped`. The queue holds at most 5000 events or 32 MB, oldest evicted first. A process dying of an uncaught exception or unhandled rejection sends the newest 16 MB it holds, blocking for at most 1 second; a graceful exit makes one final attempt capped at about 2 seconds. A run's root span is reported when the run starts, so a run whose process is killed is still listed and turns `interrupted`.
- `LOUSHO_OBSERVE` accepts `1`/`true`/`on`/`yes`, an `http(s)://` URL, or `0`/`false`/`off`/`no`/`none`/`disabled`/empty (any case). Any other value warns once naming `LOUSHO_OBSERVE` and disables reporting, instead of being posted to as a URL.
- With `LOUSHO_OBSERVE` pointing at a loopback daemon, `createAgent` captures message and tool content for the daemon by default (`captureContent: false` turns it off). Only the daemon gets it: the agent's own `exporter` (trace files, OTLP or vendor exporters, the built server's OTel export) still records no content unless the agent sets `captureContent: true`. `redactContent` does not scrub this `gen_ai.*` content; it only governs the legacy `input`/`prompt`/`args`/`result` attributes. See [Observability](docs/observability.md#message-and-argument-content-is-opt-in).
- Each observe daemon writes its own pidfile beside its db (`foo.db` gives `foo.pid`), so two dbs in one folder never share one. `lousho observe start` exits 1 when the port already serves another db, and `stop` refuses to signal when the daemon on the pidfile's port reports another db or answers as a different service. `/health` names the db it serves only to callers on the same machine. A daemon started with `--db foo.db` by an earlier build of this feature wrote the shared pidfile and is not found by the new `stop`: stop it before updating.
- `lousho observe start` and Studio print the `LOUSHO_OBSERVE` value for the port and host the daemon really uses (`LOUSHO_OBSERVE=http://127.0.0.1:7788` on a non-default port), and the Studio proxy answers `/observe/__target` with the daemon URL reporters should use.
- Agent Forge navigates with hash routes (`#/traces/<id>`: a trace has a link of its own, and back and forward work), has a left navigation instead of the Build/Observe toggle, and shows the Build controls only on Agents.
- `sandbox({ driver: 'isolate' })` runs tool code in an `isolated-vm` V8 isolate (optional peer `isolated-vm`); `defineTool({ sandbox })` opts a tool in, timeouts and OOM map to `sandbox.error` on the tool span. `'docker'` and `'e2b'` driver names are reserved for later drivers.

## [1.0.0-alpha.29] - 2026-10-21
Expand Down
8 changes: 8 additions & 0 deletions apps/agent-forge/src/router/__tests__/router.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,14 @@ describe('navigate', () => {
expect(window.location.hash).toBe('#/traces?status=error');
expect(window.location.search).toBe('?token=secret');
});
it('takes a target that already starts with # (no ##)', async () => {
navigate('#/traces/t1');
await tick();
expect(window.location.hash).toBe('#/traces/t1');
navigate('#agents');
await tick();
expect(window.location.hash).toBe('#/agents');
});
it('replace does not add a history entry', async () => {
const before = window.history.length;
navigate('/a');
Expand Down
5 changes: 3 additions & 2 deletions apps/agent-forge/src/router/router.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ export function matchRoute(hash: string, patterns: string[]): Route | null {
return null;
}

/** Go to `to` (`/traces?status=error`). `replace` swaps the current history entry. Fires `hashchange` once. */
/** Go to `to` (`/traces?status=error`, a leading `#` allowed). `replace` swaps the current history entry. Fires `hashchange` once. */
export function navigate(to: string, opts?: { replace?: boolean }): void {
const hash = `#${to.startsWith('/') ? to : `/${to}`}`;
const path = to.startsWith('#') ? to.slice(1) : to;
const hash = `#${path.startsWith('/') ? path : `/${path}`}`;
if (window.location.hash === hash) return;
if (opts?.replace) window.location.replace(hash);
else window.location.hash = hash;
Expand Down
5 changes: 3 additions & 2 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,8 +287,9 @@ hooks. Each daemon writes its own `<db name>.pid` beside the db
(`observe.db` gives `observe.pid`), so two dbs in one folder never share a
pidfile. `start` exits 1 when the port already serves a different db, and
`stop` refuses to signal when the daemon at the pidfile's port reports another
db, and falls back to signalling when it cannot verify (unreachable, or an old
one-line pidfile). See
db or the port answers as a different service, and falls back to signalling
when it cannot verify (unreachable, or an old one-line pidfile). `/health`
tells which db a daemon serves only to callers on the same machine. See
[The observe daemon](./observability.md#the-observe-daemon).

## `lousho prompt`
Expand Down
7 changes: 5 additions & 2 deletions docs/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,11 @@ names the option and `error.cause` is what the function threw.
**Means:** `createAgent({ prompt: 'name@version' })` (or `LOUSHO_PROMPT`) has no
`instructions` fallback and the prompt could not be fetched: the observe daemon
is unreachable, answered 404 for that name and version, or `LOUSHO_OBSERVE` is
not set. The first `send()` rejects and `stream()` ends with an `error` event;
the model is never called, so the agent never runs without a system prompt. The
not set. Every run rejects while the prompt stays unresolved (`send()` rejects,
`stream()` ends with an `error` event): the runs inside the 30-second retry
window at once, and each later run after its retry fetch fails while the
daemon stays down. The model is never called, so the agent never runs without
a system prompt. The
message names the prompt ref.

**Fix:** start the daemon (`lousho observe`) and set `LOUSHO_OBSERVE` to its URL,
Expand Down
2 changes: 1 addition & 1 deletion docs/evals.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ the `LOUSHO_EVAL_RESULTS` environment variable, which `lousho eval` sets and
reads back. This is more robust than a custom vitest reporter: it works across
vitest versions and worker pools, and needs no module loaded from your project.

With `LOUSHO_OBSERVE` set (or `lousho eval --observe`), each case's `eval` span and score rows are flushed to the collector when the case finishes, each wait capped at 1 second; a collector that is down or silent never fails or noticeably slows a case.
With `LOUSHO_OBSERVE` set (or `lousho eval --observe`), each case's `eval` span and score rows are flushed to the collector when the case finishes, each wait capped at 1 second; a collector that is down or silent never fails or noticeably slows a case. The score rows ride the same queue as the spans: they go out after the case's span and, while the collector is down, are kept and retried with it (and sent on exit or crash) instead of being dropped. When a wait runs out, the cases of the next 30 seconds skip the wait (their rows still queue), then the next case tries again.

### Time limits

Expand Down
32 changes: 21 additions & 11 deletions docs/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,12 +237,18 @@ the `gen_ai.*` content attributes are **never recorded by default**. Opt in
per agent with `createAgent({ exporter, captureContent: true })`.

The exception is a local collector: when `LOUSHO_OBSERVE` points at loopback
(`1`/`true`, or a URL whose host is `127.0.0.1`, `localhost` or `::1`),
(`1`/`true`, or a URL whose host is `127.0.0.1`, `localhost` or `::1`;
a `localhost` port that an SSH tunnel forwards to a remote daemon counts as
loopback too, so set `captureContent: false` if that daemon must not get content),
`createAgent` defaults `captureContent` to `true`, so the daemon's trace view
shows the prompt, the model output and the tool arguments and results that
explain a failure. The data stays in the daemon's db on your machine. Turn it
off with `captureContent: false`; a collector beyond loopback keeps the
opt-in default. The default applies to agents built with `createAgent`.
explain a failure. Only the daemon gets that content, and it stays in the
daemon's db on your machine: the agent's own `exporter` (`fileTraceExporter()`
files, an OTLP or vendor exporter, the built server's OTel export) still sees
every span without it, unless the agent sets `captureContent: true` or the
OTel content variable below is `true`. Turn it off with `captureContent: false`;
a collector beyond loopback keeps the opt-in default. The default applies to
agents built with `createAgent`.

`redactContent` keeps only the legacy `input`/`prompt`/`args`/`result`
attributes off; it does not scrub `gen_ai.*` message content. On a loopback
Expand Down Expand Up @@ -421,7 +427,8 @@ The daemon also serves two stores agents can use at runtime:
`instructions`, the run fails with `LOUSHO_PROMPT_UNRESOLVED`). A failed
fetch is not cached for good: runs in the next 30 seconds resolve at once
(using `instructions`, or failing), then the next run retries it, so a
daemon that was briefly down at build recovers without a restart. Every run logs the exact version
daemon that was briefly down at build recovers without a restart. That one
run waits for the retry fetch, up to 2.5 seconds. Every run logs the exact version
used. `lousho prompt push` publishes a new
version.
- **Context hub.** `hub()` / `hubTools()` let agents publish and query
Expand All @@ -434,16 +441,17 @@ down, the failed batch is requeued and retried with exponential backoff (1 s,
backlog drains in batches one right after another, each at most 1000 events
and 4 MB of JSON (under the daemon's per-request caps of 1000 events and
5 MB). Only these are dropped and counted in the reporter's `dropped`:
events evicted past the buffer cap (`maxBuffered`, default 5000, oldest
first); events still unsent after the exit flush's final attempt; a single
events evicted past the buffer cap (`maxBuffered`, default 5000 events, or
32 MB of serialized events, whichever comes first; oldest first); events
past the newest 16 MB when a crash flushes; events still unsent after the exit flush's final attempt; a single
event too large for any request (over 5 MB serialized, for example with
content capture on); and a batch the daemon refuses outright (`400`/`413`),
which is never retried.

A process that dies of an uncaught exception or an unhandled rejection still
reports its last run: the reporter watches `uncaughtExceptionMonitor` and
sends what it holds, oldest first, holding up the crash for up to 1 s per
reporter. Node's error output and exit code stay as they were, and a script
sends what it holds (the newest 16 MB of it), oldest first, holding up the
crash for up to 1 s per reporter. Node's error output and exit code stay as they were, and a script
that survives the throw (its own `uncaughtException` handler, or a capture
callback such as the REPL's) keeps reporting. An explicit `process.exit()` or
a SIGKILL still loses what is buffered.
Expand All @@ -465,8 +473,10 @@ several workers died), the sweep marks the quiet process's open runs
`interrupted` once it has missed about two heartbeats (the `stale` mark),
judged on that process's own heartbeat interval and on the daemon's clock. A
restart on the same host with the same pid (a container's pid 1) marks them
sooner: at the new process's second heartbeat, if the old process has been
silent since the new one appeared. With no process of the agent left, the
sooner: at the first heartbeat of the new process (after its first) that
comes more than one heartbeat interval plus 10 seconds after the old
process's last, if the old process has been silent since the new one
appeared. With no process of the agent left, the
agent goes `down` and its open runs are marked as above. A process that exits gracefully mid-run
(SIGINT, SIGTERM) marks its own open runs `interrupted` right away. An agent
with heartbeats turned off (`heartbeatIntervalMs: 0`) is never swept, so its
Expand Down
Loading
Loading