Expose GitHub CLI through the agent github tool

README.md

@@ -867,3 +867,12 @@ The agent's import_source tool also accepts branch names and defaults to the
 remote's default branch. It saves the resolved hash in conversation progress
 before invoking import-git, then attaches the snapshot and provenance at the
 requested unused path. Use /import for local checkouts.
+
+### Publishing and GitHub
+
+Ask the agent to publish a named source tree or a stack. The `publish_source` tool
+pushes the selected code commit directly from the server with an expected-head
+lease. The `github` tool runs GitHub CLI for PRs, comments, issues and linking
+existing PR URLs into stacks. Add `reader=std/github` alongside `reader=std/llm-step`
+in the `github-token` secret. See [publication and stacks](design/agent-publish.md)
+for examples, credentials and retry behavior.

design/agent-github.md

@@ -1,7 +1,8 @@
-# Importing and PRs
+# Importing and publishing
 
-Imports use three layers: the server endpoint, `caos import-git`, and the
-agent's `import_source` tool. PR publication and merge drafts come later.
+Imports are implemented through the server endpoint, `caos import-git`, and
+the agent's `import_source` tool. Publication proceeds through branches, PRs
+and stacks. Separate merge drafts remain a follow-up.
 
 ## Importing
 
@@ -46,6 +47,7 @@ Supply the token through the existing secret store:
 name=github-token
 value:@=.github-token-value
 reader=std/llm-step
+reader=std/github
 ```
 
 Keep the value file ignored and run `caos secrets` to initialize its entropy.
@@ -59,56 +61,27 @@ stay out of URLs, Git config, saved arguments, provenance, and logs. Automatic
 GitHub credentials apply only to `github.com` on the default HTTPS port.
 Public imports need no token. Importing needs neither `gh` nor another worker.
 
-## PRs
+## Publishing
 
-Add `std/github` with Git, `gh`, and a pinned
-[`gh-stack` extension](https://github.com/github/gh-stack). Give it access to
-the same `github-token` secret, exposed to `gh` as `GH_TOKEN`.
+The [publication design](agent-publish.md) separates the remaining work:
 
-Expose a general `gh` operation accepting arguments, repository, stdin, and
-input/output files. Return exit status, stdout, stderr, and requested files.
-Use a small `git_push` helper to publish a selected source commit and its history,
-requiring the remote branch to match an expected head. The server can later
-perform this transfer directly, as it does imports.
+- **Branches:** `POST /git/push`, `caos push-git` and `publish_source` push
+  exact code commits from the server with a lease. Each layer has its own
+  implementation PR.
+- **PRs:** separate PRs add the `std/github` worker and agent `github` tool. Next, validate the agent's create, update and review workflow using
+  `gh`, with explicit repositories and branch names.
+- **Stacks:** keep code boundaries in gitlinks, publish bottom to top, and
+  use `gh stack link` with existing PR URLs. Validate linking, propagation
+  and landing without a source checkout in the GitHub worker.
 
-Create PRs with explicit repository, head, and base. Inspect existing PRs before
-creating duplicates or replacing human-edited metadata. Conversation data and
-merge bookkeeping stay outside published history. Once agent publication works,
-remove `/pr`, `/publish-branch`, and their UI.
-
-### Stacks
-
-Keep each stack boundary as a source gitlink:
-
-```text
-imports/repo/base   -> H
-feature/01-core     -> A   parent H
-feature/02-tests    -> B   parent A
-```
-
-Start each layer by copying the preceding snapshot with `cp -a`, then editing
-the copy. Git ancestry records the dependency. Push A and B to corresponding
-remote branches; the first PR targets `main`, the second targets the first
-branch. If A changes, merge its new commit into B, test, and push.
-
-Link the existing PR URLs in order with
-[`gh stack link`](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands#gh-stack-link):
-
-```sh
-GH_REPO=owner/repo gh stack link --base main \
-  https://github.com/owner/repo/pull/123 \
-  https://github.com/owner/repo/pull/124
-```
-
-This needs no local stack branches. Commands such as `push`, `submit`, and
-`rebase` do require local branches and stack metadata. Supporting them would
-mean reconstructing that local repository from gitlinks and returning any
-rewritten commits to CAOS. Use CAOS's copy, edit, and merge operations initially,
-and `link` to publish the relationship. `modify` additionally requires linear
-history, so it cannot restructure stacks containing merge commits.
+The TUI's `/pr` and `/publish-branch` commands are removed. The follow-ups
+start with workflow instructions and integration tests using these tools.
 
 ### Merges
 
+This section is a follow-up proposal. Current merges still use source-tree
+conflict ledgers and the publication guards described in SPEC.md.
+
 Clean merges use the existing merge worker. On conflict, preserve the source O
 and keep the attempt beside it in the conversation:
 
@@ -138,11 +111,3 @@ is not proof that structural conflicts are resolved. Delegate by copying the
 whole attempt with `cp -a`, harvesting the edited draft, and then finishing.
 Abandoning an attempt leaves the source unchanged. Handle old source-tree
 conflict ledgers before removing their compatibility cleanup.
-
-### Retrying GitHub writes
-
-Use the tool call's durable identity to claim an operation before executing it
-and record its result afterwards. A duplicate attempt must not repeat a started
-write. After a crash or partial success, inspect GitHub before continuing;
-do not automatically retry arbitrary writes. Merge computation remains cached
-by its inputs; draft edits and completion use conditional conversation updates.

design/agent-publish.md

@@ -1,4 +1,19 @@
-# Branch publication
+# Agent publication and PR stacks
+
+Importing is implemented. Publication builds on it in three steps:
+
+| Step | What ships | Remaining work |
+| --- | --- | --- |
+| Branches | Server push endpoint, `caos push-git`, and `publish_source` in separate implementation PRs. | Merge the branch-publication implementation. |
+| PRs | The stack adds `std/github` and the agent's `github` tool in separate PRs. | Validate creating, updating and reviewing a PR through the agent. |
+| Stacks | The worker includes `gh-stack`. Source gitlinks already carry stack boundaries. | Validate publication, linking, updates and landing of a dependent stack. |
+
+The GitHub worker is tested for execution, secret grants and retry handling.
+Live PR creation and stack linking remain to be exercised. Those follow-ups
+should start with agent instructions and integration tests; the operations
+below use the existing tools.
+
+## Branches
 
 The server publishes code commits directly from its bare Git store, without
 checking out source files.
@@ -91,3 +106,192 @@ Merge conflicts currently create a tracked .caos/conflicts ledger inside a
 source tree, including conflicts without inline markers. llm-step must resolve
 and clear it before publishing. The generic endpoint does not scan for that
 ledger, inline markers, or conversation ancestry.
+
+## PRs
+
+`std/github` contains Git, `gh`, and the pinned `github/gh-stack` v0.1.1
+extension. It is registered as a built-in tool, available without a project-defined `caos-tools` entry.
+
+The tool is `github(repository, args, stdin?)`. `args` is an argument array passed
+directly to `gh`; it is never interpolated into a shell command. Return exit
+status, stdout and stderr through ordinary tool results. This covers issues,
+comments, PR creation/editing and stack operations without separate wrappers
+for each GitHub action. Initially use stdin for bodies (`--body-file -`);
+commands needing local file attachments can be added when needed.
+
+The worker sets `GH_REPO` explicitly and reads `GH_TOKEN` from its granted
+`/secret/github-token`. Add `reader=std/github` to the existing secret;
+`std/llm-step` also needs the grant: it uses the token for import/push and
+includes that identity in the model turn’s cache key. The agent carries the
+GitHub worker source and evaluates it when called, so secret marking of the
+child worker does not alter the agent’s own reader identity. Use an isolated temporary
+GitHub configuration and disable prompts. Install the extension in the image.
+The worker needs no source checkout or local branches for the operations below.
+
+### Invocation recovery
+
+Identical GitHub commands can observe different remote state, and comments
+must not be posted again when a worker retries. The harness therefore binds
+a stable, unique invocation ID into every GitHub request. A new tool call gets
+a new ID; resuming the same call retains it. Normal result caching then belongs
+to that observation, rather than to the command arguments indefinitely.
+
+A unique cache key alone does not prevent duplicate execution. Before running
+`gh`, the worker atomically claims a small record under
+`refs/caos/github/<invocation-id>` using the existing Git compare-and-swap
+transport. The record binds the entire pinned ArgTree (arguments, worker image,
+secret identity and salt) and a unique attempt ID. Recovery dispatches the
+stored task, including its original worker image. Reusing an invocation with a
+different ArgTree is an error; changed worker or secret identity requires a new
+invocation, after reconciling any earlier write. This preserves result-cache
+identity and prevents sharing results across credentials.
+
+A failed claim with no retained ref is a retryable job failure. Once gh has
+run, retry output storage and result-CAS bookkeeping up to three times, without
+rerunning gh. Exhausted bookkeeping leaves the claim uncertain.
+
+Only the attempt that owns the claim executes the command. Record the exit
+status and output hashes afterwards; completed duplicates return that result.
+
+If a claim exists without a recorded result, a duplicate reports pending or
+uncertain and does not execute `gh`. Never expire or steal that claim based on
+elapsed time. A crash between claiming and execution can therefore require
+inspection even when nothing happened. This provides at most one wrapper
+execution per invocation, not a transaction or exactly-once guarantee at GitHub.
+
+Apply this rule to all commands, including reads, to avoid classifying arbitrary
+`gh api` requests as safe or unsafe. After an uncertain write, the agent uses a
+new read invocation to inspect GitHub, then decides the remaining action.
+An absent PR or comment is not proof that a still-running command cannot create
+it. If reconciliation cannot establish the outcome, leave it uncertain rather
+than repeat the write. Failure of a multi-step command can leave partial changes.
+The tool's exit status and transcript must not claim that nothing happened.
+
+### PR workflow
+
+For a single PR:
+
+1. Integrate required updates and test the chosen source gitlink.
+2. Call `publish_source` with its path, repository and remote branch. Continue
+   after a confirmed push.
+3. Find an open PR with `gh pr list --head <branch>`; select explicitly if
+   several match. If absent, use `gh pr create --repo <repository>
+   --head <branch> --base <base> --title <title> --body-file -`, passing the
+   body through the tool's stdin. Supply these arguments explicitly so
+   creation needs no local repository.
+4. Check the PR's URL, head commit and base with `gh pr view <url> --json ...`,
+   and retain the result in the conversation.
+
+Later source edits advance the same branch through `publish_source`, updating
+the existing PR. Preserve human-edited titles and descriptions unless an edit
+was requested; use `gh pr edit` for requested metadata or base changes. Use
+`gh pr view`, `gh pr checks`, and `gh api` to read discussion, review threads
+and checks, and the corresponding CLI/API calls for requested replies.
+Create ready-for-review PRs by default. A failed PR creation leaves the
+successful branch push intact; recovery follows the invocation rules above.
+
+The PR follow-up should exercise this through the actual agent in a test
+repository: publish and open a PR, advance its head, preserve an edited body,
+read and answer review feedback, and reconcile an interrupted operation
+before proceeding. Keep repeatable failure cases in automated fixtures.
+This needs no additional server endpoint or tool for each PR action.
+
+## Stacks
+
+Keep boundaries as ordinary source gitlinks:
+
+```text
+imports/repo/base   -> H
+feature/01-core     -> A   parent H
+feature/02-tests    -> B   parent A
+```
+
+Create B by copying A with `cp -a` and editing the copy. Git ancestry establishes
+that B includes A. Publication receipts map each gitlink's published commit to
+a remote branch; GitHub holds PR URLs, bases and stack membership.
+
+Run the PR workflow bottom to top. The first PR targets the chosen mainline
+branch; each later PR targets the preceding layer's branch. Before publishing
+an upper layer, verify that it contains the exact lower commit just published.
+Initial stacks use branches in one GitHub repository.
+
+Once the PRs exist, link their URLs in order:
+
+```sh
+GH_REPO=owner/repo gh stack link --base main \
+  https://github.com/owner/repo/pull/123 \
+  https://github.com/owner/repo/pull/124
+```
+
+With `GH_REPO`, an explicit `--base`, and existing PR URLs, the pinned
+[`link` implementation](https://github.com/github/gh-stack/blob/v0.1.1/cmd/link.go)
+can use GitHub's API without a checkout or local stack tracking. Branch
+arguments take a different path that can push local branches and create PRs.
+The worker receives command arguments and PR identifiers; it does not fetch
+the source tree.
+
+`link` can change PR bases. Check the resulting bases and membership even when
+it exits with warnings. If GitHub stacks are unavailable, retain the correctly
+chained PRs and report that linking was unavailable.
+
+There is no atomic transaction spanning several pushes, PRs and stack linking.
+Record each completed step and reconcile the remainder after a partial failure;
+do not roll back successful pushes or delete PRs automatically.
+
+### Updating and landing
+
+If A changes to A1, merge A1 into B to make B1, test, then publish A1 and B1
+in that order. Both remote branches advance normally; the PRs and their URLs
+stay the same. Apply this propagation upward for additional layers.
+
+After lower PRs merge, import the actual resulting mainline tip and integrate
+it into the next surviving layer. Do not assume squash or rebase merges preserve
+A's hash. Inspect the resulting diff so already-landed changes are not proposed
+again, retarget the surviving PR if needed, and propagate the update upward.
+
+Keep clean merges and today's conflict handling initially. The separate merge
+draft gitlinks in [agent-github.md](agent-github.md#merges) remain a follow-up;
+the agent still resolves source conflicts before publishing.
+
+`gh stack link` extends existing stacks; it does not remove or reorder their
+members. Restructuring requires explicit GitHub stack changes and, when code
+dependencies change, new source commits. Do not treat another `link` call as a
+rebase or a replacement of the entire stack.
+
+### What we reuse and what remains
+
+Use `gh-stack` for GitHub's stack membership operations. The agent composes
+CAOS's existing copy, merge, test and publish operations to maintain code
+dependencies. This requires no separate stack manifest or persistent local
+branch database.
+
+[`gh stack submit`, `sync` and `rebase`](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands)
+operate on local branches and tracking state; rebasing also needs a worktree
+for code and conflicts. A bare repo containing branch refs alone would not
+make those workflows fit. Defer that adapter and history rewrites.
+
+The stack follow-up should exercise a two-layer stack in a test repository:
+publish and link it, change the lower layer and propagate upward, append a
+third layer, then land a lower PR and update the survivors. Include partial
+failure and unavailable-stack cases. Verify the actual pinned extension with
+no source checkout; version/help tests do not establish this.
+
+Start with that workflow and its tests. If an operation is missing, add the
+smallest helper it needs after checking `gh`, `gh stack` and `gh api`.
+Reordering, dropping and rebasing arbitrary layers can follow once there is
+a concrete need. Landing remains a separately requested action.
+
+## Interfaces and compatibility
+
+The agent uses publication and GitHub tools for PR operations.
+The TUI's publication commands and preview UI are removed.
+Existing publication records remain readable. Local imports keep their TUI command.
+The separate merge-draft design is deferred.
+
+Implementation: [server push endpoint](../rust/crates/server/src/push.rs),
+[agent publication](../std/llm-step/src/publish_source.rs),
+[GitHub worker](../std/github/src/main.rs), and
+[publication records](../rust/crates/conversation-protocol/src/v3/records.rs).
+External behavior: [Git push leases](https://git-scm.com/docs/git-push),
+[GitHub stack commands](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands),
+[gh-stack implementation](https://github.com/github/gh-stack).

std/llm-step/.caos-expr

@@ -22,4 +22,4 @@ CAOS_TEST=curry --base:@=DEEP-DEPS/caos-test
 CAOS_TEST_RESULT=curry --base:@=DEEP-DEPS/caos-test-result
 ASYNC_WORKER=curry --base:@=DEEP-DEPS/run-and-update-ref
 STEP=run --base:@=DEEP-DEPS/rustc --src:@=. --dep0:@=DEEP-DEPS/llm-client --dep1:@=DEEP-DEPS/conversation-protocol --dep2:@=DEEP-DEPS/git-locator --output-runner:@=DEEP-DEPS/git-runner
-curry --base=$STEP --bash-image=$BASH_TOOL --grep-image=$GREP --merge-image=$MERGE --caos-build-image=$CAOS_BUILD --caos-test-image=$CAOS_TEST --caos-test-result-image=$CAOS_TEST_RESULT --run-and-update-ref-image=$ASYNC_WORKER
+curry --base=$STEP --bash-image=$BASH_TOOL --grep-image=$GREP --merge-image=$MERGE --caos-build-image=$CAOS_BUILD --caos-test-image=$CAOS_TEST --caos-test-result-image=$CAOS_TEST_RESULT --run-and-update-ref-image=$ASYNC_WORKER --github-source:@=github

std/llm-step/github/DEPS

@@ -0,0 +1,2 @@
+# Bind this parent as data; evaluate the worker only when called.
+../../github github