From 304608982d8f7540b9272c8b22734f4c9ecdd74c Mon Sep 17 00:00:00 2001 From: archandatta <35818003+archandatta@users.noreply.github.com> Date: Fri, 25 Sep 2026 14:03:22 +0000 Subject: [PATCH 1/4] Document OTLP telemetry export Co-Authored-By: Claude Opus 5.5 --- browsers/telemetry/export.mdx | 247 ++++++++++++++++++++++++++++++++ browsers/telemetry/overview.mdx | 5 + docs.json | 3 +- 3 files changed, 254 insertions(+), 1 deletion(-) create mode 100644 browsers/telemetry/export.mdx diff --git a/browsers/telemetry/export.mdx b/browsers/telemetry/export.mdx new file mode 100644 index 00000000..80e90e5c --- /dev/null +++ b/browsers/telemetry/export.mdx @@ -0,0 +1,247 @@ +--- +title: "Export Telemetry" +description: "Send a session's captured events to your own observability backend over OTLP" +--- + +Exporting sends the events a session captures to your own OpenTelemetry backend over OTLP/HTTP, so console output, network activity, and crashes from a browser session land next to the rest of your application's logs. Export runs alongside [streaming](/browsers/telemetry/streaming) and Kernel's 30-day [retention](/browsers/telemetry/overview#retention); it doesn't replace them. + + +Events are exported as OTLP **logs**. Kernel appends `/v1/logs` to your endpoint and doesn't send traces or metrics, so set the destination up as a log source in your backend. + + +## How export works + +You create a **destination** - an OTLP/HTTP endpoint plus the headers your backend expects - and select it when you create a browser. Destinations belong to your organization, so a browser in any project can export to them. + +Kernel's relay forwards the session's events to the destination and attaches your headers at export time. Headers are encrypted at rest and never reach the browser VM, so a session can export to your backend without ever holding your ingestion key. + +## Create a destination + +Pass the base endpoint of your collector and the headers it expects. This example exports to Honeycomb: + + +```typescript Typescript/Javascript +import Kernel from '@onkernel/sdk'; + +const kernel = new Kernel(); + +const destination = await kernel.telemetry.destinations.create({ + name: 'honeycomb-prod', + endpoint: 'https://api.honeycomb.io', + description: 'Production browser telemetry', + headers: { 'x-honeycomb-team': process.env.HONEYCOMB_API_KEY! }, +}); +``` + +```python Python +import os + +from kernel import Kernel + +kernel = Kernel() + +destination = kernel.telemetry.destinations.create( + name="honeycomb-prod", + endpoint="https://api.honeycomb.io", + description="Production browser telemetry", + headers={"x-honeycomb-team": os.environ["HONEYCOMB_API_KEY"]}, +) +``` + +```bash CLI +kernel telemetry destinations create \ + --name honeycomb-prod \ + --endpoint https://api.honeycomb.io \ + --description "Production browser telemetry" \ + --header "x-honeycomb-team=$HONEYCOMB_API_KEY" +``` + + +You can also manage destinations in the dashboard under **Telemetry Destinations**. + +Creating, updating, and deleting a destination requires organization-scoped authentication, such as an organization-wide API key or the dashboard; a project-scoped API key gets a `403`. Project-scoped keys can still list and retrieve destinations, and select one when creating a browser. + +Header values are returned redacted: API, SDK, and CLI responses show each header name with an empty value. Only the dashboard shows the stored values. + +An organization can have up to 50 destinations. Names must match `^[a-zA-Z0-9._-]{1,255}$`, must be unique within the organization, and can't look like a Kernel ID. A destination can have up to 32 headers, with names and values totaling at most 8,192 bytes. + +### Endpoint rules + +The endpoint is your collector's base URL, without an OTLP signal path. Kernel appends the signal path itself, so pass `https://api.honeycomb.io`, not `https://api.honeycomb.io/v1/logs`. An endpoint that ends in `/v1/logs`, `/v1/traces`, or `/v1/metrics` is rejected. For a self-hosted OpenTelemetry Collector, use the address of its OTLP/HTTP receiver, such as `https://collector.example.com:4318`. + +The endpoint must also: + +- Use `http` or `https` +- Resolve to a public IP address +- Have no query string or fragment +- Be 2,048 characters or fewer + +Kernel doesn't follow redirects. If your endpoint responds with a redirect, the export fails and the destination reports `destination returned a redirect`, so use the final URL. + +## Export a session's telemetry + +Set `telemetry.export.otlp.destination` when you create the browser, referencing the destination by `name` or `id`: + + +```typescript Typescript/Javascript +const browser = await kernel.browsers.create({ + telemetry: { + browser: { + console: { enabled: true }, + network: { enabled: true }, + }, + export: { + otlp: { destination: { name: 'honeycomb-prod' } }, + }, + }, +}); +``` + +```python Python +browser = kernel.browsers.create( + telemetry={ + "browser": { + "console": {"enabled": True}, + "network": {"enabled": True}, + }, + "export": { + "otlp": {"destination": {"name": "honeycomb-prod"}}, + }, + }, +) +``` + +```bash CLI +kernel browsers create --telemetry=console,network --telemetry-export-otlp honeycomb-prod +``` + + +Export sends what the session captures, so capture must be on in the same request. Pass `enabled: true` or a category list alongside the destination; otherwise the request fails with `telemetry.export.otlp.destination requires telemetry capture to be enabled`. The CLI implies `--telemetry=all` when you pass `--telemetry-export-otlp` without `--telemetry`. + +Provide exactly one of `id` or `name`. Setting a destination turns export on, so you don't need to pass `enabled: true` under `otlp`, and combining a destination with `enabled: false` is rejected. + +Every captured category is exported except `screenshot` and `monitor`, whose events stay available through [streaming](/browsers/telemetry/streaming). + + +Export is bound at session creation. `browsers update` ignores `telemetry.export`, so a session keeps the destination it was created with; to export somewhere else, create a new session. Turning capture off on update also stops export, and turning capture back on resumes export to the same destination. [Browser pools](/browsers/pools) don't support export: pool create, update, and acquire reject `telemetry.export` with a `400`. + + +## What arrives in your backend + +Each event becomes one OTLP log record: + +- The record's event name is the event type, such as `network_response` or `console_error`. It's also set as the `kernel.event.type` attribute, because some backends drop the event name. +- `kernel.event.category` holds the category, and `kernel.event.seq` holds the same sequence number the [stream](/browsers/telemetry/streaming) uses. +- The body is the event's payload. +- Network events also carry `http.request.method`, `url.full`, and `http.response.status_code` when the payload has them, and console events carry `kernel.console.level`, so you can filter on them in backends that don't index a structured body. +- The resource's `service.name` is `kernel-browser`. + +See [Categories](/browsers/telemetry/categories) for every event type and what it captures. + +## Confirm export is working + +Browser responses report the session's export state under `telemetry.export.otlp`. When the session is exporting, `enabled` is `true` and `destination` is the ID of the destination it's bound to: + + +```typescript Typescript/Javascript +const session = await kernel.browsers.retrieve(browser.session_id); + +console.log(session.telemetry?.export?.otlp); +``` + +```python Python +session = kernel.browsers.retrieve(browser.session_id) + +print(session.telemetry.export.otlp) +``` + +```bash CLI +kernel browsers get -o json +``` + + +Delivery health lives on the destination itself: + + +```typescript Typescript/Javascript +const health = await kernel.telemetry.destinations.retrieve('honeycomb-prod'); + +console.log(health.last_export_at, health.consecutive_failures, health.last_error); +``` + +```python Python +health = kernel.telemetry.destinations.retrieve("honeycomb-prod") + +print(health.last_export_at, health.consecutive_failures, health.last_error) +``` + +```bash CLI +kernel telemetry destinations get honeycomb-prod +``` + + +- `last_export_at` is the time of the last successful delivery. While deliveries keep succeeding, it's refreshed about once a minute rather than on every export. +- `consecutive_failures` is `0` when the most recent recorded delivery succeeded. Read this field to tell whether the destination is failing right now. +- `last_error` and `last_error_at` describe the most recent failure and are kept after the destination recovers, so their presence alone doesn't mean exports are failing. + +`last_error` is a fixed message for the class of failure, such as `destination returned a 4xx response` or `destination TLS handshake failed`. Response bodies, endpoint URLs, and credentials are never returned. + +## Rotate credentials + +Update the destination's headers. Updates merge header by header rather than replacing the whole set, and sessions that are already exporting use the new values on their next export request, so you can rotate a key without restarting sessions: + + +```typescript Typescript/Javascript +await kernel.telemetry.destinations.update('honeycomb-prod', { + headers: { 'x-honeycomb-team': process.env.HONEYCOMB_API_KEY_NEXT! }, +}); +``` + +```python Python +kernel.telemetry.destinations.update( + "honeycomb-prod", + headers={"x-honeycomb-team": os.environ["HONEYCOMB_API_KEY_NEXT"]}, +) +``` + +```bash CLI +kernel telemetry destinations update honeycomb-prod \ + --header "x-honeycomb-team=$HONEYCOMB_API_KEY_NEXT" +``` + + +Header names are matched case-insensitively, so `authorization` replaces a stored `Authorization` instead of adding a second header. Set a header to `null` (`None` in Python, or `--remove-header` in the CLI) to delete it. Headers you don't name keep their current values. + +## Managed auth connections + +[Managed auth](/auth/overview) logins run in a browser too, so they can export the same way. Set `browser.telemetry` with an `export` block when you create or update a connection, or on a single login, using the same shape as browser create. In the CLI, pass `--telemetry-export-otlp` to `kernel auth connections create`, `update`, or `login`. + +Kernel stores the resolved destination ID on the connection, so renaming the destination later doesn't change where the connection's logins export. + +## Delete a destination + + +```typescript Typescript/Javascript +await kernel.telemetry.destinations.delete('honeycomb-prod'); +``` + +```python Python +kernel.telemetry.destinations.delete("honeycomb-prod") +``` + +```bash CLI +kernel telemetry destinations delete honeycomb-prod +``` + + +Deleting a destination fails with a `409` while it's still in use: + +- A browser session created with it hasn't ended. Wait for those sessions to end or delete them, then retry. +- A managed auth connection selects it. Point the connection at another destination or turn off its export first. +- A managed auth login using it is still in progress. Wait for the login to finish. + +## What's next + +- [Telemetry Overview](/browsers/telemetry/overview) - enable capture and choose what a session records. +- [Categories](/browsers/telemetry/categories) - every category, what it captures, and its cost characteristics. +- [Stream Telemetry](/browsers/telemetry/streaming) - consume the live stream instead of, or alongside, exporting. diff --git a/browsers/telemetry/overview.mdx b/browsers/telemetry/overview.mdx index 8fdbc0e2..f98f5b3c 100644 --- a/browsers/telemetry/overview.mdx +++ b/browsers/telemetry/overview.mdx @@ -108,6 +108,10 @@ kernel browsers update --telemetry=off On update, a category list patches the current selection - categories you don't include keep their current state. To reset the selection instead, send `enabled: true` (it replaces the selection with the categories you provide, or the default set if you provide none); send `enabled: false` to turn telemetry off. +## Exporting to your own backend + +You can also send a session's events to your own OpenTelemetry backend over OTLP/HTTP. Create a destination once for your organization, then select it with `telemetry.export.otlp.destination` when you create a browser. See [Export telemetry](/browsers/telemetry/export) for setup, limits, and how to confirm export is working. + ## Retention Captured events are retained for 30 days, then expired. You can [stream](/browsers/telemetry/streaming) events live or pull them later for analysis within that window; after 30 days the events are no longer available. @@ -116,4 +120,5 @@ Captured events are retained for 30 days, then expired. You can [stream](/browse - [Categories](/browsers/telemetry/categories) - every category, what it captures, the default set, and cost characteristics. - [Stream telemetry](/browsers/telemetry/streaming) - consume the live stream from the SDK, CLI, or raw SSE, with filtering and reconnection. +- [Export telemetry](/browsers/telemetry/export) - send captured events to your own OpenTelemetry backend over OTLP. - For event payload schemas, see the [Stream telemetry events](https://kernel.sh/docs/api-reference/browser-telemetry/stream-telemetry-events-via-sse) endpoint in the API reference. diff --git a/docs.json b/docs.json index b5cfbe00..3bb20ead 100644 --- a/docs.json +++ b/docs.json @@ -231,7 +231,8 @@ "pages": [ "browsers/telemetry/overview", "browsers/telemetry/categories", - "browsers/telemetry/streaming" + "browsers/telemetry/streaming", + "browsers/telemetry/export" ] }, "browsers/pools" From 7360a7809ec161a19afdadc58683e812d1652896 Mon Sep 17 00:00:00 2001 From: archandatta <35818003+archandatta@users.noreply.github.com> Date: Mon, 28 Sep 2026 17:24:58 +0000 Subject: [PATCH 2/4] Clarify telemetry flags for managed auth export --- browsers/telemetry/export.mdx | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/browsers/telemetry/export.mdx b/browsers/telemetry/export.mdx index 80e90e5c..9e0e219a 100644 --- a/browsers/telemetry/export.mdx +++ b/browsers/telemetry/export.mdx @@ -116,7 +116,7 @@ kernel browsers create --telemetry=console,network --telemetry-export-otlp honey ``` -Export sends what the session captures, so capture must be on in the same request. Pass `enabled: true` or a category list alongside the destination; otherwise the request fails with `telemetry.export.otlp.destination requires telemetry capture to be enabled`. The CLI implies `--telemetry=all` when you pass `--telemetry-export-otlp` without `--telemetry`. +Export sends what the session captures, so capture must be on in the same request. Pass `enabled: true` or a category list alongside the destination; otherwise the request fails with `telemetry.export.otlp.destination requires telemetry capture to be enabled`. On `kernel browsers create`, the CLI implies `--telemetry=all` when you pass a destination with `--telemetry-export-otlp` without `--telemetry`. Provide exactly one of `id` or `name`. Setting a destination turns export on, so you don't need to pass `enabled: true` under `otlp`, and combining a destination with `enabled: false` is rejected. @@ -216,6 +216,16 @@ Header names are matched case-insensitively, so `authorization` replaces a store [Managed auth](/auth/overview) logins run in a browser too, so they can export the same way. Set `browser.telemetry` with an `export` block when you create or update a connection, or on a single login, using the same shape as browser create. In the CLI, pass `--telemetry-export-otlp` to `kernel auth connections create`, `update`, or `login`. +On connection create, passing a destination implies `--telemetry=all` if you omit `--telemetry`. When selecting a destination on update or login, you must pass `--telemetry` in the same command, even if the connection already has capture enabled. This keeps the CLI from replacing your category selection with the default set: + +```bash CLI +kernel auth connections update \ + --telemetry=console,network --telemetry-export-otlp honeycomb-prod + +kernel auth connections login \ + --telemetry=console,network --telemetry-export-otlp honeycomb-prod +``` + Kernel stores the resolved destination ID on the connection, so renaming the destination later doesn't change where the connection's logins export. ## Delete a destination From e18dbbba74d922930832cae0d3ec0ac5c9df1a48 Mon Sep 17 00:00:00 2001 From: archandatta <35818003+archandatta@users.noreply.github.com> Date: Tue, 29 Sep 2026 11:19:00 +0000 Subject: [PATCH 3/4] Address review on OTLP export docs Describe dashboard header handling accurately, add a per-browser export check, and add TypeScript and Python managed auth examples. Co-Authored-By: Claude Opus 5.5 --- browsers/telemetry/export.mdx | 120 ++++++++++++++++++++++++++++++++-- 1 file changed, 113 insertions(+), 7 deletions(-) diff --git a/browsers/telemetry/export.mdx b/browsers/telemetry/export.mdx index 9e0e219a..6d73c432 100644 --- a/browsers/telemetry/export.mdx +++ b/browsers/telemetry/export.mdx @@ -61,7 +61,7 @@ You can also manage destinations in the dashboard under **Telemetry Destinations Creating, updating, and deleting a destination requires organization-scoped authentication, such as an organization-wide API key or the dashboard; a project-scoped API key gets a `403`. Project-scoped keys can still list and retrieve destinations, and select one when creating a browser. -Header values are returned redacted: API, SDK, and CLI responses show each header name with an empty value. Only the dashboard shows the stored values. +Header values are write-only. API, SDK, and CLI responses show each header name with an empty value, and the dashboard shows only header names: to change a stored value there, enter a replacement, or leave the field blank to keep the current value. An organization can have up to 50 destinations. Names must match `^[a-zA-Z0-9._-]{1,255}$`, must be unique within the organization, and can't look like a Kernel ID. A destination can have up to 32 headers, with names and values totaling at most 8,192 bytes. @@ -140,7 +140,31 @@ See [Categories](/browsers/telemetry/categories) for every event type and what i ## Confirm export is working -Browser responses report the session's export state under `telemetry.export.otlp`. When the session is exporting, `enabled` is `true` and `destination` is the ID of the destination it's bound to: +To confirm a browser's events reach your backend, generate an event you can recognize and search for it. Exported records don't carry the browser's session ID, so this example has the page log a marker that includes it: + + +```typescript Typescript/Javascript +await kernel.browsers.playwright.execute(browser.session_id, { + code: `await page.evaluate(() => console.log('kernel-export-check ${browser.session_id}'));`, +}); +``` + +```python Python +kernel.browsers.playwright.execute( + browser.session_id, + code=f"await page.evaluate(() => console.log('kernel-export-check {browser.session_id}'));", +) +``` + +```bash CLI +kernel browsers playwright execute \ + "await page.evaluate(() => console.log('kernel-export-check '))" +``` + + +Search your backend for `kernel-export-check` followed by the session ID. It arrives as a `console_log` record whose body's `text` field holds the marker. This relies on the browser capturing `console`; if it doesn't, generate an event from a category it does capture. + +If the record doesn't arrive, check the browser and the destination. Browser responses report the session's export state under `telemetry.export.otlp`. When the session is exporting, `enabled` is `true` and `destination` is the ID of the destination it's bound to: ```typescript Typescript/Javascript @@ -160,7 +184,7 @@ kernel browsers get -o json ``` -Delivery health lives on the destination itself: +Delivery health lives on the destination. It covers every session exporting to that destination, so it can show that deliveries are succeeding but not that a particular browser's events arrived: ```typescript Typescript/Javascript @@ -186,6 +210,8 @@ kernel telemetry destinations get honeycomb-prod `last_error` is a fixed message for the class of failure, such as `destination returned a 4xx response` or `destination TLS handshake failed`. Response bodies, endpoint URLs, and credentials are never returned. +Health only reflects deliveries Kernel attempted. If `telemetry.export.otlp` shows the browser is exporting, `consecutive_failures` is `0`, and your marker still hasn't arrived after a minute, [contact support](mailto:support@kernel.sh) with the session ID. + ## Rotate credentials Update the destination's headers. Updates merge header by header rather than replacing the whole set, and sessions that are already exporting use the new values on their next export request, so you can rotate a key without restarting sessions: @@ -214,17 +240,97 @@ Header names are matched case-insensitively, so `authorization` replaces a store ## Managed auth connections -[Managed auth](/auth/overview) logins run in a browser too, so they can export the same way. Set `browser.telemetry` with an `export` block when you create or update a connection, or on a single login, using the same shape as browser create. In the CLI, pass `--telemetry-export-otlp` to `kernel auth connections create`, `update`, or `login`. +[Managed auth](/auth/overview) logins run in a browser too, so they can export the same way. Set `browser.telemetry` with an `export` block when you create or update a connection, or on a single login, using the same shape as browser create: + + +```typescript Typescript/Javascript +const auth = await kernel.auth.connections.create({ + domain: 'example.com', + profile_name: 'my-profile', + browser: { + telemetry: { + browser: { + console: { enabled: true }, + network: { enabled: true }, + }, + export: { + otlp: { destination: { name: 'honeycomb-prod' } }, + }, + }, + }, +}); +``` -On connection create, passing a destination implies `--telemetry=all` if you omit `--telemetry`. When selecting a destination on update or login, you must pass `--telemetry` in the same command, even if the connection already has capture enabled. This keeps the CLI from replacing your category selection with the default set: +```python Python +auth = kernel.auth.connections.create( + domain="example.com", + profile_name="my-profile", + browser={ + "telemetry": { + "browser": { + "console": {"enabled": True}, + "network": {"enabled": True}, + }, + "export": { + "otlp": {"destination": {"name": "honeycomb-prod"}}, + }, + }, + }, +) +``` ```bash CLI -kernel auth connections update \ - --telemetry=console,network --telemetry-export-otlp honeycomb-prod +kernel auth connections create \ + --domain example.com \ + --profile-name my-profile \ + --telemetry=console,network \ + --telemetry-export-otlp honeycomb-prod +``` + + +A `browser.telemetry` block on update or login replaces the connection's stored telemetry config instead of merging into it, so include the categories you want captured along with the export block. On login, the block applies to that login only, and the connection keeps its stored config: + + +```typescript Typescript/Javascript +const login = await kernel.auth.connections.login(auth.id, { + browser: { + telemetry: { + browser: { + console: { enabled: true }, + network: { enabled: true }, + }, + export: { + otlp: { destination: { name: 'honeycomb-prod' } }, + }, + }, + }, +}); +``` + +```python Python +login = kernel.auth.connections.login( + auth.id, + browser={ + "telemetry": { + "browser": { + "console": {"enabled": True}, + "network": {"enabled": True}, + }, + "export": { + "otlp": {"destination": {"name": "honeycomb-prod"}}, + }, + }, + }, +) +``` +```bash CLI kernel auth connections login \ --telemetry=console,network --telemetry-export-otlp honeycomb-prod ``` + + +`kernel auth connections update` takes the same flags as `login`. On connection create, passing a destination implies `--telemetry=all` if you omit `--telemetry`. When selecting a destination on update or login, you must pass `--telemetry` in the same command, even if the connection already has capture enabled. This keeps the CLI from replacing your category selection with the default set. Kernel stores the resolved destination ID on the connection, so renaming the destination later doesn't change where the connection's logins export. From 84260f039e31aa4c03d105cb7b7fee055291f7d8 Mon Sep 17 00:00:00 2001 From: archandatta <35818003+archandatta@users.noreply.github.com> Date: Thu, 1 Oct 2026 13:31:09 +0000 Subject: [PATCH 4/4] Split dense export caveats into bullets Co-Authored-By: Claude Opus 5.5 --- browsers/telemetry/export.mdx | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/browsers/telemetry/export.mdx b/browsers/telemetry/export.mdx index 6d73c432..6cc11018 100644 --- a/browsers/telemetry/export.mdx +++ b/browsers/telemetry/export.mdx @@ -123,7 +123,11 @@ Provide exactly one of `id` or `name`. Setting a destination turns export on, so Every captured category is exported except `screenshot` and `monitor`, whose events stay available through [streaming](/browsers/telemetry/streaming). -Export is bound at session creation. `browsers update` ignores `telemetry.export`, so a session keeps the destination it was created with; to export somewhere else, create a new session. Turning capture off on update also stops export, and turning capture back on resumes export to the same destination. [Browser pools](/browsers/pools) don't support export: pool create, update, and acquire reject `telemetry.export` with a `400`. +Export is bound at session creation: + +- **Destination is fixed:** `browsers update` ignores `telemetry.export`, so a session keeps the destination it was created with. To export somewhere else, create a new session. +- **Capture controls export:** turning capture off on update also stops export, and turning capture back on resumes export to the same destination. +- **No browser pools:** [browser pools](/browsers/pools) don't support export. Pool create, update, and acquire reject `telemetry.export` with a `400`. ## What arrives in your backend @@ -330,7 +334,11 @@ kernel auth connections login \ ``` -`kernel auth connections update` takes the same flags as `login`. On connection create, passing a destination implies `--telemetry=all` if you omit `--telemetry`. When selecting a destination on update or login, you must pass `--telemetry` in the same command, even if the connection already has capture enabled. This keeps the CLI from replacing your category selection with the default set. +In the CLI: + +- `kernel auth connections update` takes the same flags as `login`. +- On connection create, passing a destination implies `--telemetry=all` if you omit `--telemetry`. +- When selecting a destination on update or login, you must pass `--telemetry` in the same command, even if the connection already has capture enabled. This keeps the CLI from replacing your category selection with the default set. Kernel stores the resolved destination ID on the connection, so renaming the destination later doesn't change where the connection's logins export.