Skip to content

deploy-doctor and health-digest, on Claude Code headless, as piabrota-ops - #1

Merged
h0ffmann merged 2 commits into
mainfrom
claude/agents-deploy-doctor-and-health-digest
Oct 31, 2026
Merged

h0ffmann merged 2 commits into
mainfrom
claude/agents-deploy-doctor-and-health-digest

Conversation

@h0ffmann

@h0ffmann h0ffmann commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

What

The first two agents, run on the VPS as rax tasks (the pod runner from piabrota/rax#10, the runtime from piabrota/nix-os#36), on the Claude subscription (CLAUDE_CODE_OAUTH_TOKEN from claude setup-token), posting as the GitHub App piabrota-ops.

Agent Reads Writes
deploy-doctor each failed deploy run of nix-os in the last day with no issue yet: the failed jobs' logs (last 400 lines), the run, the repository at that commit one issue per run, labelled deploy-failure: verdict (transient, real, unclear), what failed, why with the log lines, what to do. With no new failure it exits without calling Claude
health-digest the box's snapshot (ConfigMap ax-tasks/health-snapshot, or HEALTH_SNAPSHOT_FILE), the last 10 deploys, the previous digest a daily comment on the open health-digest issue (opened the first time): status, services, memory and disk with the change, pods, deploys, errors, what to watch
  • bin/agent: the entrypoint; a task's command is agent deploy-doctor or agent health-digest.
  • lib/github-app.sh: the App's JWT (RS256 with openssl) and installation tokens for one named repository with named permissions, refusing an empty set, as marola-devkit's bot-token (MIP-0086). deploy-doctor gets actions read, contents read, issues write; health-digest actions read, issues write.
  • lib/common.sh: the GitHub API, labels, and claude -p with --allowedTools limited to read-only tools, --strict-mcp-config, --no-session-persistence.
  • prompts/: each agent's prompt, with the shape of its answer.
  • Dockerfile: Debian slim, curl jq openssl git, Claude Code's standalone binary 2.1.316 pinned by its SHA-256 for amd64 and arm64, user 10001, auto-updater and telemetry off. 525 MB.
  • .github/workflows/ci.yml: shellcheck and test/run.sh. image.yml: publishes ghcr.io/piabrota/agents as rax's does.
  • README: the agents, the env keys, the App's setup and permissions, the tests.

Tests

  • test/run.sh, 23 checks: both agents against a fake GitHub API (test/fake_github.py, standard library) and a fake claude. The JWT verifies against the App's public key and is issued by its ID; the token is for nix-os alone with the agent's permissions; later calls carry the installation token; Claude Code gets the run, only the failed job's log and the repository (doctor) or the snapshot, deploys and previous digest (digest), read-only, on sonnet or haiku, without MCP; one issue per failed run, never twice, without calling Claude the second time; the digest issue is opened once and reused, and the next day sees yesterday's digest; the guards (permissions required, a missing token named with where it comes from).
  • The image built (arm64 here): runs as uid 10001, claude --version is 2.1.316, agent health-digest without its Secret says what is missing.
  • Not tested: a real run, which needs the token and the App's key on the VPS. The first one after nix-os pins this image is the test.

Next, in nix-os

Pin this image's digest; the tasks and their CronJobs (every 15 minutes for the doctor, daily for the digest); the box's health snapshot as the ConfigMap, and the service account that reads it.

Second commit: a strict harness, PR comments, signatures, code-smell CI

  • Layer 1, what Claude can do: --restricted (no Bash or other code-running tool, no WebFetch, no settings files, file tools confined to the agent's directory), --tools read-only only, --strict-mcp-config, pinned model IDs (claude-sonnet-5-5, claude-haiku-5-5), timeout.
  • Layer 2, what it returns: --json-schema per agent (prompts/*.schema.json: enums, required fields, lengths); the house rules prompts/_rules.md appended to every run; the Markdown rendered by the harness from the JSON.
  • Layer 3, what gets posted: the doctor checks every quoted evidence line is in the file it names and every named file exists; one retry told what failed; then an unclear issue with only the harness's facts. rules/transient.txt (SSH timeouts, cache errors, a lost runner) fixes the verdict at transient before Claude runs. Credentials are scrubbed before posting. A digest that misses its schema twice fails the run instead of posting.
  • PR comments: the doctor comments on the merged pull request a deploy came from (verdict, what to do, link to the issue). Needs Pull requests: read and write on piabrota-ops; without it, the issue still comes and the comment is skipped, with a log line.
  • Signature on every report: <sub>agent · model via Claude Code version · turns, seconds · as user (uid) in pod · OS · kernel · vCPUs (CPU) · RAM, memory limit · image</sub>.
  • .github/workflows/code-smell.yml: ruff 0.14.2 with every rule (ruff.toml), radon 6.0.1 failing on rank C or worse, shfmt 3.10.0 (checksum pinned) on the bash. test/fake_github.py became a route table: worst complexity B (7), was E (31).
  • Tests: 31 checks, among them evidence not in the log rejected and retried, a missing file twice giving the facts-only issue, a rule overriding the verdict, tokens scrubbed, the PR comment, the App without pull_requests, the digest failing without posting.

…-ops

Two bash agents for piabrota/nix-os, run as rax tasks on the VPS. deploy-doctor
opens one issue per failed deploy run with Claude Code's diagnosis of its logs
and the repository at that commit; health-digest comments daily on a
health-digest issue from the box's snapshot, the last deploys and the previous
digest. Both post as the GitHub App piabrota-ops, with tokens minted per run
for nix-os alone and only the permissions each needs, as marola-devkit's
bot-token does. Claude Code runs with read-only tools, no MCP, on the
subscription's CLAUDE_CODE_OAUTH_TOKEN.

The image pins Claude Code's standalone binary by checksum; test/run.sh runs
both agents against a fake GitHub and a fake claude.
…sign reports

Layer 1: Claude Code runs --restricted, with --tools limited to read-only ones,
no MCP, a pinned model ID and a time limit. Layer 2: --json-schema per agent,
the house rules (prompts/_rules.md) appended to every run, and the Markdown
rendered from the JSON. Layer 3: the doctor checks that quoted evidence is in
the file it names and named files exist, retries once with what failed, and
otherwise posts only the facts as unclear; rules/transient.txt fixes the verdict
of known transient failures before Claude runs; credentials are scrubbed.

deploy-doctor also comments on the merged pull request a deploy came from
(pull_requests:write; without it the issue still comes). Every report ends
with a signature: agent, model, Claude Code version, turns and time, and where
it ran.

code-smell.yml adds ruff with every rule and radon (rank C fails) for the
Python, and shfmt for the bash; the fake GitHub is now a route table.
@h0ffmann
h0ffmann merged commit 32f1e1e into main Oct 31, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant