diff --git a/README.md b/README.md index aad75b5cb..320f108a5 100644 --- a/README.md +++ b/README.md @@ -867,3 +867,12 @@ The agent's import_source tool also accepts branch names and defaults to the remote's default branch. It saves the resolved hash in conversation progress before invoking import-git, then attaches the snapshot and provenance at the requested unused path. Use /import for local checkouts. + +### Publishing and GitHub + +Ask the agent to publish a named source tree or a stack. The `publish_source` tool +pushes the selected code commit directly from the server with an expected-head +lease. The `github` tool runs GitHub CLI for PRs, comments, issues and linking +existing PR URLs into stacks. Add `reader=std/github` alongside `reader=std/llm-step` +in the `github-token` secret. See [publication and stacks](design/agent-publish.md) +for examples, credentials and retry behavior. diff --git a/design/agent-github.md b/design/agent-github.md index 4afc1df38..163ab6dc4 100644 --- a/design/agent-github.md +++ b/design/agent-github.md @@ -1,7 +1,8 @@ -# Importing and PRs +# Importing and publishing -Imports use three layers: the server endpoint, `caos import-git`, and the -agent's `import_source` tool. PR publication and merge drafts come later. +Imports are implemented through the server endpoint, `caos import-git`, and +the agent's `import_source` tool. Publication proceeds through branches, PRs +and stacks. Separate merge drafts remain a follow-up. ## Importing @@ -46,6 +47,7 @@ Supply the token through the existing secret store: name=github-token value:@=.github-token-value reader=std/llm-step +reader=std/github ``` Keep the value file ignored and run `caos secrets` to initialize its entropy. @@ -59,56 +61,27 @@ stay out of URLs, Git config, saved arguments, provenance, and logs. Automatic GitHub credentials apply only to `github.com` on the default HTTPS port. Public imports need no token. Importing needs neither `gh` nor another worker. -## PRs +## Publishing -Add `std/github` with Git, `gh`, and a pinned -[`gh-stack` extension](https://github.com/github/gh-stack). Give it access to -the same `github-token` secret, exposed to `gh` as `GH_TOKEN`. +The [publication design](agent-publish.md) separates the remaining work: -Expose a general `gh` operation accepting arguments, repository, stdin, and -input/output files. Return exit status, stdout, stderr, and requested files. -Use a small `git_push` helper to publish a selected source commit and its history, -requiring the remote branch to match an expected head. The server can later -perform this transfer directly, as it does imports. +- **Branches:** `POST /git/push`, `caos push-git` and `publish_source` push + exact code commits from the server with a lease. Each layer has its own + implementation PR. +- **PRs:** separate PRs add the `std/github` worker and agent `github` tool. Next, validate the agent's create, update and review workflow using + `gh`, with explicit repositories and branch names. +- **Stacks:** keep code boundaries in gitlinks, publish bottom to top, and + use `gh stack link` with existing PR URLs. Validate linking, propagation + and landing without a source checkout in the GitHub worker. -Create PRs with explicit repository, head, and base. Inspect existing PRs before -creating duplicates or replacing human-edited metadata. Conversation data and -merge bookkeeping stay outside published history. Once agent publication works, -remove `/pr`, `/publish-branch`, and their UI. - -### Stacks - -Keep each stack boundary as a source gitlink: - -```text -imports/repo/base -> H -feature/01-core -> A parent H -feature/02-tests -> B parent A -``` - -Start each layer by copying the preceding snapshot with `cp -a`, then editing -the copy. Git ancestry records the dependency. Push A and B to corresponding -remote branches; the first PR targets `main`, the second targets the first -branch. If A changes, merge its new commit into B, test, and push. - -Link the existing PR URLs in order with -[`gh stack link`](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands#gh-stack-link): - -```sh -GH_REPO=owner/repo gh stack link --base main \ - https://github.com/owner/repo/pull/123 \ - https://github.com/owner/repo/pull/124 -``` - -This needs no local stack branches. Commands such as `push`, `submit`, and -`rebase` do require local branches and stack metadata. Supporting them would -mean reconstructing that local repository from gitlinks and returning any -rewritten commits to CAOS. Use CAOS's copy, edit, and merge operations initially, -and `link` to publish the relationship. `modify` additionally requires linear -history, so it cannot restructure stacks containing merge commits. +The TUI's `/pr` and `/publish-branch` commands are removed. The follow-ups +start with workflow instructions and integration tests using these tools. ### Merges +This section is a follow-up proposal. Current merges still use source-tree +conflict ledgers and the publication guards described in SPEC.md. + Clean merges use the existing merge worker. On conflict, preserve the source O and keep the attempt beside it in the conversation: @@ -138,11 +111,3 @@ is not proof that structural conflicts are resolved. Delegate by copying the whole attempt with `cp -a`, harvesting the edited draft, and then finishing. Abandoning an attempt leaves the source unchanged. Handle old source-tree conflict ledgers before removing their compatibility cleanup. - -### Retrying GitHub writes - -Use the tool call's durable identity to claim an operation before executing it -and record its result afterwards. A duplicate attempt must not repeat a started -write. After a crash or partial success, inspect GitHub before continuing; -do not automatically retry arbitrary writes. Merge computation remains cached -by its inputs; draft edits and completion use conditional conversation updates. diff --git a/design/agent-publish.md b/design/agent-publish.md index 8f8144765..3971cc1dc 100644 --- a/design/agent-publish.md +++ b/design/agent-publish.md @@ -1,4 +1,19 @@ -# Branch publication +# Agent publication and PR stacks + +Importing is implemented. Publication builds on it in three steps: + +| Step | What ships | Remaining work | +| --- | --- | --- | +| Branches | Server push endpoint, `caos push-git`, and `publish_source` in separate implementation PRs. | Merge the branch-publication implementation. | +| PRs | The stack adds `std/github` and the agent's `github` tool in separate PRs. | Validate creating, updating and reviewing a PR through the agent. | +| Stacks | The worker includes `gh-stack`. Source gitlinks already carry stack boundaries. | Validate publication, linking, updates and landing of a dependent stack. | + +The GitHub worker is tested for execution, secret grants and retry handling. +Live PR creation and stack linking remain to be exercised. Those follow-ups +should start with agent instructions and integration tests; the operations +below use the existing tools. + +## Branches The server publishes code commits directly from its bare Git store, without checking out source files. @@ -91,3 +106,192 @@ Merge conflicts currently create a tracked .caos/conflicts ledger inside a source tree, including conflicts without inline markers. llm-step must resolve and clear it before publishing. The generic endpoint does not scan for that ledger, inline markers, or conversation ancestry. + +## PRs + +`std/github` contains Git, `gh`, and the pinned `github/gh-stack` v0.1.1 +extension. It is registered as a built-in tool, available without a project-defined `caos-tools` entry. + +The tool is `github(repository, args, stdin?)`. `args` is an argument array passed +directly to `gh`; it is never interpolated into a shell command. Return exit +status, stdout and stderr through ordinary tool results. This covers issues, +comments, PR creation/editing and stack operations without separate wrappers +for each GitHub action. Initially use stdin for bodies (`--body-file -`); +commands needing local file attachments can be added when needed. + +The worker sets `GH_REPO` explicitly and reads `GH_TOKEN` from its granted +`/secret/github-token`. Add `reader=std/github` to the existing secret; +`std/llm-step` also needs the grant: it uses the token for import/push and +includes that identity in the model turn’s cache key. The agent carries the +GitHub worker source and evaluates it when called, so secret marking of the +child worker does not alter the agent’s own reader identity. Use an isolated temporary +GitHub configuration and disable prompts. Install the extension in the image. +The worker needs no source checkout or local branches for the operations below. + +### Invocation recovery + +Identical GitHub commands can observe different remote state, and comments +must not be posted again when a worker retries. The harness therefore binds +a stable, unique invocation ID into every GitHub request. A new tool call gets +a new ID; resuming the same call retains it. Normal result caching then belongs +to that observation, rather than to the command arguments indefinitely. + +A unique cache key alone does not prevent duplicate execution. Before running +`gh`, the worker atomically claims a small record under +`refs/caos/github/` using the existing Git compare-and-swap +transport. The record binds the entire pinned ArgTree (arguments, worker image, +secret identity and salt) and a unique attempt ID. Recovery dispatches the +stored task, including its original worker image. Reusing an invocation with a +different ArgTree is an error; changed worker or secret identity requires a new +invocation, after reconciling any earlier write. This preserves result-cache +identity and prevents sharing results across credentials. + +A failed claim with no retained ref is a retryable job failure. Once gh has +run, retry output storage and result-CAS bookkeeping up to three times, without +rerunning gh. Exhausted bookkeeping leaves the claim uncertain. + +Only the attempt that owns the claim executes the command. Record the exit +status and output hashes afterwards; completed duplicates return that result. + +If a claim exists without a recorded result, a duplicate reports pending or +uncertain and does not execute `gh`. Never expire or steal that claim based on +elapsed time. A crash between claiming and execution can therefore require +inspection even when nothing happened. This provides at most one wrapper +execution per invocation, not a transaction or exactly-once guarantee at GitHub. + +Apply this rule to all commands, including reads, to avoid classifying arbitrary +`gh api` requests as safe or unsafe. After an uncertain write, the agent uses a +new read invocation to inspect GitHub, then decides the remaining action. +An absent PR or comment is not proof that a still-running command cannot create +it. If reconciliation cannot establish the outcome, leave it uncertain rather +than repeat the write. Failure of a multi-step command can leave partial changes. +The tool's exit status and transcript must not claim that nothing happened. + +### PR workflow + +For a single PR: + +1. Integrate required updates and test the chosen source gitlink. +2. Call `publish_source` with its path, repository and remote branch. Continue + after a confirmed push. +3. Find an open PR with `gh pr list --head `; select explicitly if + several match. If absent, use `gh pr create --repo + --head --base --title --body-file -`, passing the + body through the tool's stdin. Supply these arguments explicitly so + creation needs no local repository. +4. Check the PR's URL, head commit and base with `gh pr view <url> --json ...`, + and retain the result in the conversation. + +Later source edits advance the same branch through `publish_source`, updating +the existing PR. Preserve human-edited titles and descriptions unless an edit +was requested; use `gh pr edit` for requested metadata or base changes. Use +`gh pr view`, `gh pr checks`, and `gh api` to read discussion, review threads +and checks, and the corresponding CLI/API calls for requested replies. +Create ready-for-review PRs by default. A failed PR creation leaves the +successful branch push intact; recovery follows the invocation rules above. + +The PR follow-up should exercise this through the actual agent in a test +repository: publish and open a PR, advance its head, preserve an edited body, +read and answer review feedback, and reconcile an interrupted operation +before proceeding. Keep repeatable failure cases in automated fixtures. +This needs no additional server endpoint or tool for each PR action. + +## Stacks + +Keep boundaries as ordinary source gitlinks: + +```text +imports/repo/base -> H +feature/01-core -> A parent H +feature/02-tests -> B parent A +``` + +Create B by copying A with `cp -a` and editing the copy. Git ancestry establishes +that B includes A. Publication receipts map each gitlink's published commit to +a remote branch; GitHub holds PR URLs, bases and stack membership. + +Run the PR workflow bottom to top. The first PR targets the chosen mainline +branch; each later PR targets the preceding layer's branch. Before publishing +an upper layer, verify that it contains the exact lower commit just published. +Initial stacks use branches in one GitHub repository. + +Once the PRs exist, link their URLs in order: + +```sh +GH_REPO=owner/repo gh stack link --base main \ + https://github.com/owner/repo/pull/123 \ + https://github.com/owner/repo/pull/124 +``` + +With `GH_REPO`, an explicit `--base`, and existing PR URLs, the pinned +[`link` implementation](https://github.com/github/gh-stack/blob/v0.1.1/cmd/link.go) +can use GitHub's API without a checkout or local stack tracking. Branch +arguments take a different path that can push local branches and create PRs. +The worker receives command arguments and PR identifiers; it does not fetch +the source tree. + +`link` can change PR bases. Check the resulting bases and membership even when +it exits with warnings. If GitHub stacks are unavailable, retain the correctly +chained PRs and report that linking was unavailable. + +There is no atomic transaction spanning several pushes, PRs and stack linking. +Record each completed step and reconcile the remainder after a partial failure; +do not roll back successful pushes or delete PRs automatically. + +### Updating and landing + +If A changes to A1, merge A1 into B to make B1, test, then publish A1 and B1 +in that order. Both remote branches advance normally; the PRs and their URLs +stay the same. Apply this propagation upward for additional layers. + +After lower PRs merge, import the actual resulting mainline tip and integrate +it into the next surviving layer. Do not assume squash or rebase merges preserve +A's hash. Inspect the resulting diff so already-landed changes are not proposed +again, retarget the surviving PR if needed, and propagate the update upward. + +Keep clean merges and today's conflict handling initially. The separate merge +draft gitlinks in [agent-github.md](agent-github.md#merges) remain a follow-up; +the agent still resolves source conflicts before publishing. + +`gh stack link` extends existing stacks; it does not remove or reorder their +members. Restructuring requires explicit GitHub stack changes and, when code +dependencies change, new source commits. Do not treat another `link` call as a +rebase or a replacement of the entire stack. + +### What we reuse and what remains + +Use `gh-stack` for GitHub's stack membership operations. The agent composes +CAOS's existing copy, merge, test and publish operations to maintain code +dependencies. This requires no separate stack manifest or persistent local +branch database. + +[`gh stack submit`, `sync` and `rebase`](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands) +operate on local branches and tracking state; rebasing also needs a worktree +for code and conflicts. A bare repo containing branch refs alone would not +make those workflows fit. Defer that adapter and history rewrites. + +The stack follow-up should exercise a two-layer stack in a test repository: +publish and link it, change the lower layer and propagate upward, append a +third layer, then land a lower PR and update the survivors. Include partial +failure and unavailable-stack cases. Verify the actual pinned extension with +no source checkout; version/help tests do not establish this. + +Start with that workflow and its tests. If an operation is missing, add the +smallest helper it needs after checking `gh`, `gh stack` and `gh api`. +Reordering, dropping and rebasing arbitrary layers can follow once there is +a concrete need. Landing remains a separately requested action. + +## Interfaces and compatibility + +The agent uses publication and GitHub tools for PR operations. +The TUI's publication commands and preview UI are removed. +Existing publication records remain readable. Local imports keep their TUI command. +The separate merge-draft design is deferred. + +Implementation: [server push endpoint](../rust/crates/server/src/push.rs), +[agent publication](../std/llm-step/src/publish_source.rs), +[GitHub worker](../std/github/src/main.rs), and +[publication records](../rust/crates/conversation-protocol/src/v3/records.rs). +External behavior: [Git push leases](https://git-scm.com/docs/git-push), +[GitHub stack commands](https://docs.github.com/en/pull-requests/reference/stacked-prs-cli-commands), +[gh-stack implementation](https://github.com/github/gh-stack). diff --git a/std/llm-step/.caos-expr b/std/llm-step/.caos-expr index 307b46aa1..9c9871afb 100644 --- a/std/llm-step/.caos-expr +++ b/std/llm-step/.caos-expr @@ -22,4 +22,4 @@ CAOS_TEST=curry --base:@=DEEP-DEPS/caos-test CAOS_TEST_RESULT=curry --base:@=DEEP-DEPS/caos-test-result ASYNC_WORKER=curry --base:@=DEEP-DEPS/run-and-update-ref STEP=run --base:@=DEEP-DEPS/rustc --src:@=. --dep0:@=DEEP-DEPS/llm-client --dep1:@=DEEP-DEPS/conversation-protocol --dep2:@=DEEP-DEPS/git-locator --output-runner:@=DEEP-DEPS/git-runner -curry --base=$STEP --bash-image=$BASH_TOOL --grep-image=$GREP --merge-image=$MERGE --caos-build-image=$CAOS_BUILD --caos-test-image=$CAOS_TEST --caos-test-result-image=$CAOS_TEST_RESULT --run-and-update-ref-image=$ASYNC_WORKER +curry --base=$STEP --bash-image=$BASH_TOOL --grep-image=$GREP --merge-image=$MERGE --caos-build-image=$CAOS_BUILD --caos-test-image=$CAOS_TEST --caos-test-result-image=$CAOS_TEST_RESULT --run-and-update-ref-image=$ASYNC_WORKER --github-source:@=github diff --git a/std/llm-step/github/DEPS b/std/llm-step/github/DEPS new file mode 100644 index 000000000..a49e0838a --- /dev/null +++ b/std/llm-step/github/DEPS @@ -0,0 +1,2 @@ +# Bind this parent as data; evaluate the worker only when called. +../../github github diff --git a/std/llm-step/src/github.rs b/std/llm-step/src/github.rs new file mode 100644 index 000000000..57e738da7 --- /dev/null +++ b/std/llm-step/src/github.rs @@ -0,0 +1,196 @@ +//! The GitHub worker is independent of source-tree materialization. +use super::*; + +pub(super) fn declaration() -> Value { + json!({ + "name":"github", + "description":"Run GitHub CLI with literal arguments and an explicit owner/repository. Supports issues, comments, PRs and remote stack operations. Every new tool call observes GitHub afresh; retries never repeat an unfinished invocation. After an uncertain write, use a new read call to inspect the outcome; an absent result does not prove an in-flight write failed. For PRs, first publish_source, then find/create a PR with explicit --head and --base; preserve existing human-edited titles/bodies. Create ready PRs unless a draft was requested. For stacks, publish source gitlinks bottom to top, each including the exact published lower commit. Use gh stack link --base <mainline> with existing PR URLs in order; verify bases and membership afterwards. Link can extend a stack, not remove or reorder it. submit/sync/rebase require local branches and are not supported by this worker. Multi-step failures can leave partial changes.", + "input_schema":{"type":"object","additionalProperties":false,"properties":{ + "repository":{"type":"string","description":"GitHub owner/repository."}, + "args":{"type":"array","items":{"type":"string"},"minItems":1,"description":"Arguments after gh, e.g. [\"pr\",\"list\",\"--head\",\"feature/a\",\"--json\",\"url,baseRefName,headRefOid\"]."}, + "stdin":{"type":"string","description":"Optional standard input; pass bodies with --body-file -."} + },"required":["repository","args"]} + }) +} + +#[derive(serde::Deserialize)] +#[serde(deny_unknown_fields)] +struct Parameters { + repository: String, + args: Vec<String>, + stdin: Option<String>, +} + +pub(super) fn start( + cfg: &Config, + state: &mut progress::State, + site: &CallSite<'_>, +) -> Result<bool, String> { + if cfg.github_source.is_none() { + site.fail(state, "GitHub worker is unavailable")?; + return Ok(true); + } + // The enclosing model turn must also be isolated by the GitHub identity. + // Its grant is already required for import_source and publish_source. + if secret("github-token").is_err() { + site.fail( + state, + "Grant github-token to both std/llm-step and std/github before using GitHub tools", + )?; + return Ok(true); + } + let me = self_curry( + None, + site.request, + site.round, + &site.call.id, + &[ + ("current-tool", Arg::Lit("github")), + ("tool-eval", Arg::Lit("github")), + ], + )?; + // Keep source data in the agent image. Embedding a secret-marked worker + // changes its bindings and prevents the agent's own reader grant matching. + eval_then_catching(&arg("github-source"), "DEEP-DEPS/github", Arg::Hash(&me))?; + Ok(false) +} + +pub(super) fn evaluated( + cfg: &Config, + state: &mut progress::State, + request: &Oid, + request_head: &Oid, + round: u64, + id: &str, +) -> Result<(), String> { + let view = state.conversation()?; + let record = require_request(&view, request)?; + let current = round_state(&view, &record)?; + if record.status != TurnStatus::Running + || current.declaring_round != round + || view.tool(request, round, id)?.is_some() + { + return resume(cfg, state, request, request_head); + } + let call = current + .pending + .iter() + .find(|call| call.id == id) + .cloned() + .ok_or("evaluated GitHub call is no longer pending")?; + let site = CallSite::at(request, round, &call, ¤t.declaration_message); + if Path::new(&arg("error")).exists() { + site.fail(state, &read_arg("error")?)?; + return resume(cfg, state, request, request_head); + } + let image = cas_hash(&arg("result"))?; + if launch(&image, state, &site)? { + resume(cfg, state, request, request_head) + } else { + Ok(()) + } +} + +fn launch(image: &str, state: &mut progress::State, site: &CallSite<'_>) -> Result<bool, String> { + let p: Parameters = match serde_json::from_value(site.call.input.clone()) { + Ok(p) => p, + Err(error) => { + site.fail(state, &format!("invalid github arguments: {error}"))?; + return Ok(true); + } + }; + if git_locator::github_repository(&p.repository).is_err() + || p.args.is_empty() + || p.args.iter().any(|a| a.contains('\0')) + { + site.fail( + state, + "github requires owner/repository and a nonempty array of literal arguments", + )?; + return Ok(true); + } + let invocation = publish_source::invocation(&state.conversation()?.identity()?.id, site)?; + let args = serde_json::to_string(&p.args).map_err(|e| e.to_string())?; + let mut bindings = vec![ + ("repository", Arg::Lit(&p.repository)), + ("args", Arg::Lit(&args)), + ("invocation", Arg::Lit(&invocation)), + ]; + if let Some(stdin) = p.stdin.as_deref() { + bindings.push(("stdin", Arg::Lit(stdin))); + } + let task = Oid::parse( + &caos_curry(Arg::Hash(image), &bindings)?, + "GitHub invocation", + )?; + let mut record = site.stub(None); + record.status = CallStatus::Started; + record.task = Some(task); + let head = state.head().clone(); + match state.try_append_at( + &head, + Transition::ToolStart { + record: record.clone(), + payloads: Vec::new(), + }, + )? { + progress::TryAppend::HeadChanged(_) => Ok(true), + progress::TryAppend::Appended(_) => { + dispatch(site.request, site.round, site.call, &record)?; + Ok(false) + } + } +} + +pub(super) fn dispatch( + request: &Oid, + round: u64, + call: &Call, + record: &CallRecord, +) -> Result<(), String> { + let me = self_curry( + None, + request, + round, + &call.id, + &[("current-tool", Arg::Lit("github"))], + )?; + // Let the server assemble the request and attach its secret grant. + // The worker ignores this immutable definition passed as run-then's input. + worker_common::run_then_catching( + &arg("github-source"), + Arg::Hash( + record + .task + .as_ref() + .ok_or("GitHub start has no task")? + .as_str(), + ), + Arg::Hash(&me), + ) +} + +pub(super) fn result(record: &CallRecord) -> Result<(Value, Option<Oid>), String> { + let text = read_arg("result")?; + let value: Value = serde_json::from_str(&text).map_err(|_| "invalid GitHub worker result")?; + let error = value["status"] != "complete" || value["exit"] != 0; + // The full blob remains addressable through the normal tool result. + let text = if text.len() > 100_000 { + let mut start = text.len() - 100_000; + while !text.is_char_boundary(start) { + start += 1; + } + format!( + "Output truncated; showing the last 100000 bytes:\n{}", + &text[start..] + ) + } else { + text + }; + let text = if value["status"] == "uncertain" { + format!("UNCERTAIN: the command may have changed GitHub. Inspect with a new read call before taking further action; do not resend the write.\n{text}") + } else { + text + }; + Ok((result_block(&record.id, &text, error), None)) +} diff --git a/std/llm-step/src/main.rs b/std/llm-step/src/main.rs index 7243fc5d5..9614bcb25 100644 --- a/std/llm-step/src/main.rs +++ b/std/llm-step/src/main.rs @@ -2,6 +2,7 @@ mod async_work; mod githist; +mod github; mod import_source; mod progress; mod publish_source; @@ -60,6 +61,7 @@ struct Config { bash_image: String, grep_image: Option<String>, merge_image: Option<String>, + github_source: Option<String>, std_tool_images: BTreeMap<&'static str, Option<String>>, run_and_update_ref_image: Option<String>, /// Drain this request's declared calls and STOP -- do not call the model, @@ -124,6 +126,7 @@ impl Config { bash_image: image_arg("bash-image")?.ok_or("--bash-image is required")?, grep_image: image_arg("grep-image")?, merge_image: image_arg("merge-image")?, + github_source: image_arg("github-source")?, std_tool_images: STD_TOOLS .iter() .map(|&(name, argument)| Ok((name, image_arg(argument)?))) @@ -297,6 +300,9 @@ fn callback( timing::phase(&format!("tool wait {tool}")); if read_arg_opt("tool-eval")?.is_some() { + if tool == "github" { + return github::evaluated(cfg, state, request, request_head, round, &id); + } if Path::new(&arg("error")).exists() { let error = read_arg("error")?; let block = failed_run_block(&id, &tool, &error); @@ -1073,6 +1079,9 @@ fn drive_call( return Ok(true); } + if call.name == "github" { + return github::start(cfg, state, &site); + } if call.name == subagents::SPAWN_TOOL { return spawn_agent_call(cfg, state, &site); } @@ -1623,6 +1632,9 @@ fn dispatch_started( call: &Call, record: &CallRecord, ) -> Result<(), String> { + if call.name == "github" { + return github::dispatch(request, round, call, record); + } let commit = record .input_commit .as_ref() @@ -1688,6 +1700,7 @@ fn callback_result( record: &CallRecord, ) -> Result<(Value, Option<Oid>), String> { match record.name.as_str() { + "github" => github::result(record), subagents::WAIT_TOOL => wait_callback_block(state, record), "grep" => { let scope = read_arg_opt("scope")?.unwrap_or_default(); @@ -2971,6 +2984,9 @@ fn registry(cfg: &Config) -> Result<Vec<Value>, String> { registry.push(with_source_tree(merge_tool())); } registry.extend(githist::declarations().into_iter().map(with_source_tree)); + if cfg.github_source.is_some() { + registry.push(github::declaration()); + } for &(name, arg_name) in &STD_TOOLS { if cfg.std_tool_images.get(name).is_some_and(Option::is_some) { if let Some(tool) = tools::std_tool(name, &arg(arg_name))? { diff --git a/std/llm-step/src/source_trees.rs b/std/llm-step/src/source_trees.rs index acae55c89..eaeeb8ecc 100644 --- a/std/llm-step/src/source_trees.rs +++ b/std/llm-step/src/source_trees.rs @@ -13,7 +13,7 @@ Conversation filesystem: - Before integrating a publication base, compare the intended feature change with the full source-versus-destination difference. Merging upstream preserves all existing branch changes; rebasing a whole branch may replay inherited changes too. Transplanting only the requested edit onto a new base is a separate operation. If a small task would publish unrelated inherited work, explain that scope and ask whether to retain the branch or transplant the edit. Do not claim that targeting an older upstream isolates the edit. Preserve existing snapshots when making a new starting point. - Resolve source-tree conflicts by editing affected files and clearing their ledger entries. Saving an edited source tree removes an empty .caos/conflicts ledger and prunes its .caos directory if empty. Unresolved entries and other metadata stay intact. Do not recreate a removed ledger to register resolution. This source-tree ledger is distinct from protected conversation-root .caos. - Preparing, delegating, merging, testing, and organizing a PR stack requires no publishing destination. Preserve the imported starting commit at 00-base. Finish the requested work without asking for repository publication settings. -- When branch publication is requested, use publish_source with an explicit gitlink, HTTPS repository and branch. Inspect the complete PR diff, incorporate the chosen base and prior stack boundary, resolve conflicts and run checks first. No publication policy file or local branch database is needed. Import provenance can identify the repository and default base; ask only when ambiguous. +- When publication is requested, use publish_source with an explicit gitlink, HTTPS repository and branch. Inspect the complete PR diff, incorporate the chosen base and prior stack boundary, resolve conflicts and run checks first. Use github for PR lookup/creation and stack linking. No publication policy file or local branch database is needed. Import provenance can identify the repository and default base; ask only when ambiguous. - If a tool fails, report the observed error and uncertainty; do not invent storage behavior or claim success from a failed check. Use available repository tools for relevant tests; describe a specific missing capability rather than assuming workers cannot run tests or reach a server. - Git tools require the target commit-entry path explicitly. Invoke repository tools with run_tool at a conversation-relative path. UI selection never changes your execution context. @@ -21,11 +21,14 @@ Conversation filesystem: - Use import_source(source="https://github.com/owner/repo.git", revision="main", into="imports/repo/main-2") for remote code. Omit revision for the default branch. Public imports need no token; private imports use the granted github-token secret. Each new call observes the remote; retries of a persisted call keep its pinned commit. Choose an unused path and preserve the imported snapshot. The tool returns a full commit hash and records provenance in its .source.json sibling. It does not merge code. - For origin/main, read the selected source's import provenance, use that repository URL and revision main with import_source, then merge the returned commit into the intended source tree. If provenance is ambiguous, use an explicit repository. Remote names are not local ref snapshots. +- Publish stack layers bottom to top. Each upper source commit must include the exact published lower commit. Find the PR by repository and head branch before creating one; name the base branch explicitly, preserve existing titles and bodies, and create ready-for-review PRs unless drafts were requested. Link existing PR URLs with github args ["stack","link","--base",mainline,lowerUrl,upperUrl]; verify bases and membership afterwards. If native stack linking is unavailable, retain the valid chained PRs and explain the limitation. Never roll back successful pushes after a later step fails. +- When a lower layer changes, merge it into each upper layer, test, and publish upward. After a lower PR lands, import actual mainline (squash/rebase may change hashes), integrate it and inspect the surviving diff before retargeting. gh stack submit/sync/rebase need local branches and are deferred; force pushes are unavailable. Publishing does not authorize landing PRs. + Client actions (give these instructions to the user, not to bash or run_tool): - When a task needs a client action, give the exact TUI command or keys and concrete conversation path. Do not just say "ask to import", "trigger an import", or "publish it". Use supplied paths/URLs; ask only for missing information. Do not claim a client operation succeeded without its result. Subagents report client needs to their parent. - Local import syntax: /import <conversation-path> <local-Git-checkout> [revision]. For example: /import imports/repo/base /path/to/repo main. Omit revision only for a clean checkout's HEAD. Local paths refer to the machine running the TUI, relative to its launch directory or absolute; ~ and environment variables are not expanded. Quote spaces. Give the user this exact command when host code is needed; after importing, they send a message to continue. - To inspect files, offer Ctrl+O and name the path to navigate to. The browser shows adjacent-boundary diffs, not the PR diff against a remote base. Names stay unchanged when their commits advance. Browser selection does not choose a publication or checkout target. -- For a PR, finish preparing and checking the changes, then give /pr <conversation/gitlink> <base-remote-branch> [remote-URL], e.g. /pr feature/01-parser main. Omit the URL when matching import provenance identifies it; otherwise supply a known URL or ask only for what is missing. Use a known base branch (import metadata may provide the default), never guess one. The command previews the source hash, repository, branch, and base; it does not display a commit graph or full PR diff. Enter confirms pushing and opening or updating that PR. For a stack, suggest one command per boundary in order, e.g. /pr feature/02-errors feature/01-parser after the first PR is published. Each command publishes exactly its named snapshot; sibling names do not choose the PR base. /publish-branch <conversation/gitlink> [remote-URL] pushes a branch without creating a PR. The source and base must share Git history. If the source lacks the current base tip, the preview offers to import it and ask you to merge or rebase and test. That confirmation does not publish; finish the requested integration and suggest /pr again. If the base branch is missing, correct the command or publish the preceding PR first. Do not request publication settings merely to prepare changes. Do not suggest Ctrl+P. + - For editing on the host, give /checkout <conversation/gitlink> <local-directory>, e.g. /checkout feature/01-parser /path/to/checkout. Omitting the directory reuses that gitlink's remembered destination. This checks out a detached commit. Use /update-tree <conversation/gitlink> <message> to bring edits from that path's remembered checkout back into the conversation and continue. Always name the gitlink; browser selection does not choose the target. Ctrl+L is not a checkout shortcut. These client commands do not change your tool paths. "#; diff --git a/std/llm-step/src/tools.rs b/std/llm-step/src/tools.rs index 7ed757d33..9dbec7391 100644 --- a/std/llm-step/src/tools.rs +++ b/std/llm-step/src/tools.rs @@ -113,6 +113,7 @@ pub fn grep_declaration() -> Value { const RESERVED_TOOLS: &[&str] = &[ "import_source", "publish_source", + "github", "bash", "grep", "read", diff --git a/tests/README.md b/tests/README.md index 08eb47519..9bf83c785 100644 --- a/tests/README.md +++ b/tests/README.md @@ -23,6 +23,6 @@ The llm-import entry exercises agent-driven importing in an empty conversation, including provenance, readable history, occupied destinations, and invalid sources. Pinning and replay are also covered by llm-step's unit tests. -The github entry exercises the packaged CLI, secret grants and invocation -replay using the version command. It requires +The github entry exercises the packaged CLI and stack extension, secret grants, +invocation replay and the agent callback using version/help commands. It requires no GitHub account and makes no external writes. diff --git a/tests/github/DEPS b/tests/github/DEPS index 071997e85..de60570cb 100644 --- a/tests/github/DEPS +++ b/tests/github/DEPS @@ -1,2 +1,4 @@ ../../dev/cli-test cli-test ../../std/github github +../../std/llm-step llm-step +../../std/llm-stub llm-stub diff --git a/tests/github/worker.sh b/tests/github/worker.sh index 1f1a1f3c9..abfa3441c 100755 --- a/tests/github/worker.sh +++ b/tests/github/worker.sh @@ -2,12 +2,14 @@ set -euo pipefail fail() { echo "FAIL: $*" >&2; exit 1; } -# No GitHub account or external mutation: version exercises the real image, -# secret grant and invocation record. +# No GitHub account or external mutation: version/help exercise the real image, +# extension, secret grant, invocation record, and agent callback. mkdir -p .caos-secrets printf '.caos-secrets/\n' >> .git/info/exclude printf '%s\n' 'value=github-fixture-token' 'entropy=0123456789abcdef0123456789abcdef' \ - 'reader=DEEP-DEPS/github' > .caos-secrets/github-token + 'reader=DEEP-DEPS/github' 'reader=DEEP-DEPS/llm-step' > .caos-secrets/github-token +printf '%s\n' 'value=model-fixture-token' 'entropy=abcdef0123456789abcdef0123456789' \ + 'reader=DEEP-DEPS/llm-step' > .caos-secrets/anthropic-api-key id=$(printf '%s' "$(date +%s%N)-$$-$RANDOM" | sha256sum) id=${id%% *} "$CAOS_CLI" run version --base:@=DEEP-DEPS/github --repository=owner/repo \ @@ -22,4 +24,30 @@ if "$CAOS_CLI" run changed --base:@=DEEP-DEPS/github --repository=owner/repo \ fi grep -q "different request" changed.err || fail "unclear invocation error" +"$CAOS_CLI" get DEEP-DEPS/llm-stub /tmp/stub-entry +install -m 755 /tmp/stub-entry/bin/llm-stub /tmp/github-llm-stub +mkdir stub +printf '%s\n' '{"content":[{"type":"tool_use","id":"gh","name":"github","input":{"repository":"owner/repo","args":["stack","link","--help"]}}],"stop_reason":"tool_use"}' > stub/response-1.json +printf '%s\n' '{"content":[{"type":"text","text":"checked"}],"stop_reason":"end_turn"}' > stub/response-2.json +port=$((20000 + RANDOM % 20000)) +/tmp/github-llm-stub "0.0.0.0:$port" "$PWD/stub" 2>stub/log & +stub_pid=$! +trap 'kill "$stub_pid" 2>/dev/null || true' EXIT +ready=0 +for _ in $(seq 1 400); do + if (exec 3<>"/dev/tcp/127.0.0.1/$port") 2>/dev/null; then ready=1; break; fi + sleep 0.01 +done +[ "$ready" = 1 ] || fail "stub did not start" +"$CAOS_CLI" chat "github-$id" -m "Check stack link help." --username tester --model test-model \ + --base-url "http://${CAOS_STUB_HOST:-host.containers.internal}:$port" \ + --llm-step:@=DEEP-DEPS/llm-step +jq -e '.tools | any(.name == "github")' stub/request-1.json >/dev/null +jq -e '.tools[] | select(.name == "publish_source") | .input_schema | + (.properties.source_tree.type == "string") and + (.required | index("source_tree") != null)' stub/request-1.json >/dev/null +jq -e '[.messages[].content | select(type == "array") | .[] | + select(.type == "tool_result" and .tool_use_id == "gh")] | + length == 1 and .[0].is_error != true and + (.[0].content | map(.text) | join("") | contains("gh stack link"))' stub/request-2.json >/dev/null echo "github: ALL PASS"