feature/Runner-log-context · L-260916-b7759f — drop shim, json sink, no rich - #81
Conversation
… boots on the json sink Every value an error line carries is a record attribute now instead of a token in a rendered `key=value` run: the handlers hand their fields to the runtime's log call as `fields=`, `RequestIdMiddleware` binds `request_id` on the runtime's own log context so it lands on every record emitted during a request — the ones pipelex emits from inside a run included — and `.pipelex/pipelex.toml` selects the `json` sink on stderr with a Rich-free pretty-print mode. The API's logfmt flattener and its escaping go with the shim: a structured sink serializes a value itself, so the message is now a short sentence built only from the status and the error type, and no caller-supplied string reaches it. `route` could not go where the item's text put it. `log.context` accepts exactly three run identifiers — `request_id`, `pipeline_run_id`, `pipe_run_id` — and the runtime is explicit that they are a closed reserved set spelled to match the payload fields they come from. A route path is not one of them, so it travels as a field on the call sites that want it, which is also the simplest thing: every one of those sites already holds the `Request`. `api/logging_context.py` is gone rather than rewritten. Its two contextvars also fed `instance` and `request_id` into the problem documents the `api.errors` helpers build, and those two members are now stamped by the handler that renders the response, which is the frame that holds the `Request` — the same values, from the same place, as the pipelex-error and request-validation paths already used. The helpers stay parameter-clean and the API keeps no ambient request state. The log seam is in no published pipelex release, so `pipelex` resolves from a git source declared in `[tool.uv.sources]` for the duration of the sprint; the pin site says what collapses it. Only uv reads that entry, so it steers the lock, the sync and the Docker build and nothing else. One adjacent fix, found while trying to run the image: `.dockerignore` did not exclude `.pipelex/*_override.toml`, so a local `make docker-build` baked the builder's own untracked overrides — storage backend, log level, telemetry credentials — into the image, which then behaved differently from the one CI builds out of a clean checkout. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…his server is
Round 1 of the review confirmed seven findings, and what they have in common is
that the change described itself more confidently than it behaved.
The image context excluded one tier of per-developer configuration where the
runtime layers several: `_local`, `_{environment}` and `_temporary_override` too,
for every configuration family rather than for `pipelex` alone, plus two nested
under `inference/`. Environment names are open-ended, so the exclusion is now by
shape, with the one tracked variant named back in. Verified by building a
throwaway context and listing what survived.
The configuration test read the merged config while its comment claimed it pinned
the shipped file. The merged config is partly the developer's: a local
`sink = "console"` reds the suite while CI, which has no override file, stays
green. It reads the shipped file now, which is what the image boots on.
Three places said a boot selecting the console sink or the `rich` pretty-print
mode would refuse at startup. It would not: `typer` and `instructor` are core
pipelex dependencies that require Rich unconditionally, so Rich is in the image
and reachable, and these keys are the whole of what keeps it unused. The
Dockerfile already said this correctly and now the others agree with it.
`detail` was documented as a field every error line carries, but only the
failures this API authors itself attach it — a Pipelex `ErrorReport` has been
through disclosure redaction, so its body text is not the cause and is
deliberately not logged. Said so, in the page, the changelog and the repo guide.
The documented "replace the entire config directory" mount drops the shipped
`pipelex.toml` and silently returns the server to Pipelex's terminal-facing
defaults, which is precisely what this work exists to avoid. It now warns, and
names the keys to re-supply.
And the boot refuses a `pipelex` without `log.context`, by name. The branch the
pin names declares the same version as the published release, so no version
specifier can distinguish them and a `pip install .` resolves the wrong one;
without the check its first symptom is the error handler raising on the first
failure it tries to report.
Advances L-260916-b7759f
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…sal that names the cause is reached A published pipelex refuses this server's [runtime.log] sink key while api.main is being imported, so the check in lifespan never ran on the install it exists for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vmb35cM2bCxTwn49kCAXQ
…oes red TestClient runs the app in a portal thread whose context never reaches the test's, so the release test passed with a binding that was never released. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vmb35cM2bCxTwn49kCAXQ
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vmb35cM2bCxTwn49kCAXQ
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019vmb35cM2bCxTwn49kCAXQ
…text # Conflicts: # pyproject.toml # uv.lock
`pipelex-api` takes `pipelex` from `pipelex`'s feature/Rich-behind-extra, written at [tool.uv.sources] of pyproject.toml with the lock regenerated in the same commit (P1). The collapse before this branch merges is `wt unpin _pipelex-api--runner-log-context pipelex --to <version>` (P2, P7).
`pipelex-api` takes `pipelex` from `pipelex`'s feature/Rich-behind-extra, written at [tool.uv.sources] of pyproject.toml with the lock regenerated in the same commit (P1). The collapse before this branch merges is `wt unpin _pipelex-api--runner-log-context pipelex --to <version>` (P2, P7).
…text Absorbs dev's v0.26.1 through v0.27.5. The manifest keeps the branch's sprint git source for `pipelex` over dev's `==0.65.0` pin, the lock keeps that source and takes dev's package version, and the changelog keeps the branch's Unreleased section above every released section dev added. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpAMq6PZNQfauyDvx3GR2H
`pipelex-api` takes `pipelex` from `pipelex`'s feature/Rich-behind-extra, written at [tool.uv.sources] of pyproject.toml with the lock regenerated in the same commit (P1). The collapse before this branch merges is `wt unpin _pipelex-api--runner-log-context pipelex --to <version>` (P2, P7).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpAMq6PZNQfauyDvx3GR2H
|
Four reviewers ran: cubic, Codex review, Codex adversarial and code-review at level medium. code-review found nothing. Two findings were verified: Codex review's claim that Nothing was fixed, because the bar admits only criticals and there were none. Every other finding is deferred with its trace in converged — the round-4 pass at profile 4 was clean with full coverage, at or above the required depth of 4; nothing further is warranted |
`pipelex-api` takes `pipelex` from the registry again, with the lock regenerated in the same commit. The pin stood at 80896463 of `pipelex` (P7).
`wt unpin` replaced the git source and re-locked; this finishes the collapse. The requirement is the exact `==0.66.0` again, as this dependency is pinned everywhere else in the history, and the comments in `pyproject.toml` and the Dockerfile that explained the git source are gone with it. So is the runtime-contract guard, which refused a non-`uv` install resolving a published `pipelex` and was written to be deleted with the source, along with its test. The release also carries the redaction processor, which escapes a control character in a field's value before any sink sees the record, so the crafted-detail test now expects the printable escape, a new test pins that a credential echoed into `detail` is redacted, and `docs/logging.md` documents both along with the trace-context keys. The changelog's temporary git-source entry becomes the pin entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TpAMq6PZNQfauyDvx3GR2H
The runner now binds the runtime's log context per request and emits its fields through the runtime's log calls, so its own request-id contextvars and logfmt flattener are gone. It boots on the json sink with the silent pretty-print mode and builds its image without the Rich extra; it pins
pipelex0.66.0, the release that carries the log seam.Closes L-260916-b7759f
🤖 Generated with Claude Code
https://claude.ai/code/session_019vmb35cM2bCxTwn49kCAXQ
Summary by cubic
Replaces the runner's request-id contextvars and logfmt flattener with Pipelex's runtime log context and a JSON line sink on stderr. Error lines now carry each value as its own JSON field, and
request_idis bound per request so every record the request emits — including ones Pipelex emits from inside a run — carries it. Drops thecliextra, so the Rich terminal renderer is never reached.Closes L-260916-b7759f.
Notes
pipelex.tomlmust keepsink = "json",console_log_target = "stderr", andpretty_print_mode = "silent".pipelexhas collapsed onto the released0.66.0; the runtime-contract guard and its test are deleted with it.pipelex0.66.0's redaction processor escapes control characters in field values and redacts credentials echoed intodetail; tests anddocs/logging.mdcover both..pipelexoverride files into the image.wip/structured-logs/review-deferrals.md.Written for commit ef292eb. Summary will update on new commits.