Reject gitignored files before branch publication

design/agent-publish.md

@@ -36,6 +36,7 @@ The endpoint performs one push:
    stored commit. Trust the complete history verified at ingestion and startup.
 2. If E is non-null, require E to be a stored ancestor of H. If H contains E,
    CAOS already has E. A missing E or non-fast-forward is a rejection.
+   Reject H if its tree contains paths matched by its own .gitignore rules.
 3. Push H to the destination branch with
    --force-with-lease=refs/heads/<branch>:<E>, disabling tag following.
    Empty E requires creation. The ancestry check prevents history rewrites;
@@ -45,8 +46,8 @@ The endpoint performs one push:
    per-ref receiver rejections are definite failures. Unconfirmed transport
    failures are uncertain. The CLI preserves these results for llm-step.
 
-The endpoint does not fetch, import, merge, rebase, resolve source policy, or
-perform a follow-up remote lookup. Objects transfer directly from CAOS to the
+The endpoint does not fetch, import, merge, rebase, rewrite commits, or perform
+a follow-up remote lookup. Objects transfer directly from CAOS to the
 destination through Git. Reuse import authentication: token-file option,
 sensitive header and repository-scoped credential helper.
 
@@ -63,19 +64,28 @@ reason through the CLI to the tool result. Importing and integration are
 never hidden inside a push.
 
 A remote can accept a push before the connection drops. Git's HTTP retry can
-then report a stale lease. After a conflict or uncertain result, llm-step reads
-the branch: H confirms completion. Otherwise it preserves a definite rejection;
+then report a stale lease. After a receiver conflict or uncertain result,
+llm-step reads the branch: H confirms completion. Otherwise it preserves a definite rejection;
 for uncertainty, another value than E is a conflict, while E or a failed lookup
 remains uncertain because a push may still be running. The endpoint itself does no recovery. A success receipt records the
 original push even if the branch later advances.
 
-Publication transfers the exact commit, including its tracked files. It does
-not apply .gitignore or remove .caos content. Local Git staging respects
-.gitignore for untracked files, and host path ingestion uses git ls-files.
-Remote imports preserve their existing commit. The agent's bash tool, however,
-stores every real file left in its working tree through caos put; that path
-currently does not apply .gitignore. Ignore handling belongs in source capture
-as a follow-up, before a commit is formed, not in publication.
+Publication transfers the exact commit. Before pushing, the server checks H's
+tree against its versioned .gitignore files, including nested rules and
+negations. A match returns HTTP 422 with code `ignored-files`; the CLI and
+agent retain this as a definite rejection without remote reconciliation. Git
+performs the check using a private index; no source files are checked out.
+
+This is deliberately stricter than ordinary Git: a tracked file matching an
+ignore rule is rejected too. Global excludes and .git/info/exclude do not
+apply. This checks the requested snapshot only, not earlier commits; a file
+added and deleted in its history is outside this check. The server never
+strips files or rewrites history.
+
+Local Git staging still respects .gitignore for untracked files. Imports keep
+their exact commits, and agent tools continue to capture files as they do
+today. Ignored scratch files can therefore remain in a source during work;
+the agent must remove them or adjust the rules before publication.
 
 Merge conflicts currently create a tracked .caos/conflicts ledger inside a
 source tree, including conflicts without inline markers. llm-step must resolve

rust/crates/git-locator/src/publish.rs

@@ -5,6 +5,7 @@ pub fn diagnostic(code: &str) -> Option<&'static str> {
         "missing-commit" => "The server does not hold the source commit. Import it before publishing.",
         "missing-expected" => "The server does not hold the expected remote head. Import it before publishing.",
         "not-fast-forward" => "The update is not a fast-forward. Import and merge the remote head before publishing.",
+        "ignored-files" => "The source commit contains files matched by its .gitignore rules. Remove those files or adjust the ignore rules before publishing; no push was attempted.",
         "validation-failed" => "The server could not validate the source commit; no push was attempted.",
         "lease-rejected" => "The remote head does not match the pinned lease. Inspect it before importing and integrating changes.",
         "hook-declined" => "The remote rejected the push: a receive hook declined it. Check repository rules and branch protection.",

rust/crates/server/src/push.rs

@@ -84,7 +84,7 @@ pub(crate) fn endpoint(
         .truncate(false)
         .read(true)
         .write(true)
-        .open(locks.join(key))?;
+        .open(locks.join(&key))?;
     lock.lock()?;
 
     let receipt =
@@ -110,6 +110,16 @@ pub(crate) fn endpoint(
             _ => return Err(reject("validation-failed")),
         }
     }
+    if ignored_files(
+        &input.destination,
+        &config.git_dir,
+        &input.commit,
+        &locks.join(format!("{key}.check")),
+    )
+    .map_err(|_| reject("validation-failed"))?
+    {
+        return Err(reject("ignored-files"));
+    }
     let lease = format!(
         "--force-with-lease={refname}:{}",
         input.expected.as_deref().unwrap_or("")
@@ -161,6 +171,85 @@ pub(crate) fn endpoint(
     }
 }
 
+// The destination lock owns this scratch directory, including leftovers after
+// a crash. Only an index and path list are written: skip-worktree lets Git read
+// nested .gitignore blobs from the index without checking out source files.
+fn ignored_files(
+    destination: &str,
+    git_dir: &str,
+    commit: &str,
+    directory: &std::path::Path,
+) -> Result<bool, HttpError> {
+    use std::fs::{self, File};
+    if directory.exists() {
+        fs::remove_dir_all(directory)?;
+    }
+    fs::create_dir_all(directory.join("work"))?;
+    let directory = fs::canonicalize(directory)?;
+    let result = (|| {
+        let invalid = || HttpError::new(422, "Git ignore validation failed");
+        let run = |args: &[&str], stdin: Option<File>| -> Result<Vec<u8>, HttpError> {
+            let mut command = git_locator::import::git(destination, None).map_err(|_| invalid())?;
+            command
+                .args(["--git-dir", git_dir, "-c", "core.bare=false"])
+                .args(["-c", "core.sparseCheckout=false", "--work-tree"])
+                .arg(directory.join("work"))
+                .env("GIT_INDEX_FILE", directory.join("index"))
+                .args(args);
+            if let Some(input) = stdin {
+                command.stdin(input);
+            }
+            let output = command.output().map_err(|_| invalid())?;
+            if !output.status.success() {
+                return Err(invalid());
+            }
+            Ok(output.stdout)
+        };
+        run(&["read-tree", commit], None)?;
+        // Only regular ignore files may supply patterns. Marking a symlink
+        // skip-worktree would make Git's index fallback parse its link target.
+        let mut patterns = Vec::new();
+        for entry in run(&["ls-files", "--stage", "-z"], None)?
+            .split(|b| *b == 0)
+            .filter(|entry| entry.starts_with(b"100644 ") || entry.starts_with(b"100755 "))
+        {
+            let path = entry
+                .splitn(2, |b| *b == b'\t')
+                .nth(1)
+                .ok_or_else(invalid)?;
+            if path == b".gitignore" || path.ends_with(b"/.gitignore") {
+                patterns.extend_from_slice(path);
+                patterns.push(0);
+            }
+        }
+        if patterns.is_empty() {
+            return Ok(false);
+        }
+        let paths = directory.join("paths");
+        fs::write(&paths, patterns)?;
+        run(
+            &["update-index", "--skip-worktree", "-z", "--stdin"],
+            Some(File::open(paths)?),
+        )?;
+        // Explicit per-directory rules exclude host/global/info/exclude policy.
+        // --cached deliberately checks tracked entries too: this is a publication
+        // rule, stricter than Git's ordinary admission of untracked files.
+        Ok(!run(
+            &[
+                "ls-files",
+                "--cached",
+                "--ignored",
+                "--exclude-per-directory=.gitignore",
+                "-z",
+            ],
+            None,
+        )?
+        .is_empty())
+    })();
+    fs::remove_dir_all(directory)?;
+    result
+}
+
 // Only a per-ref porcelain rejection proves the receiver refused this update.
 // Missing status (including authentication/transport failures) remains uncertain.
 fn rejection(stdout: &[u8], refname: &str) -> Option<&'static str> {

std/llm-step/src/main.rs

@@ -4401,11 +4401,15 @@ mod tests {
                 .unwrap();
             assert_eq!(recovered, pending);
             let outcome = if rejected {
-                conversation_protocol::v3::publication::Outcome::new(
-                    PublicationStatus::Conflict,
-                    "validation-rejected",
-                    Some("Invalid source".into()),
-                    None,
+                publish_source::reconcile(
+                    &recovered,
+                    conversation_protocol::v3::publication::Outcome::new(
+                        PublicationStatus::Conflict,
+                        "validation-rejected",
+                        Some("Source contains ignored files".into()),
+                        None,
+                    ),
+                    || panic!("validation rejection must not observe the remote"),
                 )
             } else {
                 // Git may resend after a lost acknowledgement and report a
@@ -4418,7 +4422,7 @@ mod tests {
                         None,
                         None,
                     ),
-                    Ok(Some(commit.clone())),
+                    || Ok(Some(commit.clone())),
                 )
             };
             publish_source::finish(&mut state, &site, &pending, Some(outcome)).unwrap();

std/llm-step/src/publish_source.rs

@@ -3,7 +3,7 @@ use super::*;
 use conversation_protocol::v3::publication::Outcome;
 use conversation_protocol::v3::{Descriptor, PublicationRecord, PublicationStatus};
 
-pub(super) const HELP: &str = "Publish the exact selected source commit to an HTTPS Git repository branch, preserving its history. Test and inspect the intended PR diff first. Resolve merge conflicts and clear .caos/conflicts before publishing. The endpoint pushes the commit unchanged; it does not filter files or apply .gitignore. This does not create a PR or change the source gitlink. Only fast-forward updates are supported: import and merge remote changes before retrying a conflict. A receipt names the exact published commit even if the source later changes. On uncertainty, inspect the remote before taking another action.
+pub(super) const HELP: &str = "Publish the exact selected source commit to an HTTPS Git repository branch, preserving its history. Test and inspect the intended PR diff first. Resolve merge conflicts and clear .caos/conflicts before publishing. The endpoint rejects files matched by the source commit's .gitignore rules, including tracked files. Remove those files or adjust the rules before publishing. It never strips files or rewrites commits. This does not create a PR or change the source gitlink. Only fast-forward updates are supported: import and merge remote changes before retrying a conflict. A receipt names the exact published commit even if the source later changes. On uncertainty, inspect the remote before taking another action.
 @param repository HTTPS Git repository URL, without credentials.
 @param branch Destination branch name (without refs/heads/).";
 
@@ -172,11 +172,7 @@ pub(super) fn execute(state: &mut progress::State, site: &CallSite<'_>) -> Resul
                         value["diagnostic"].as_str().map(str::to_owned),
                         observed,
                     );
-                    if status == PublicationStatus::Complete {
-                        outcome
-                    } else {
-                        reconcile(&pending, outcome, observe())
-                    }
+                    reconcile(&pending, outcome, observe)
                 }
                 Ok(output) if output.status.code() == Some(1) => Outcome::new(
                     PublicationStatus::Conflict,
@@ -203,9 +199,16 @@ pub(super) fn invocation(conversation: &str, site: &CallSite<'_>) -> Result<Stri
 pub(super) fn reconcile(
     pending: &PublicationRecord,
     outcome: Outcome,
-    observed: Result<Option<Oid>, String>,
+    observe: impl FnOnce() -> Result<Option<Oid>, String>,
 ) -> Outcome {
-    match observed {
+    // A local/server validation refusal means no push was attempted. A remote
+    // head that already matches must not hide the rejection or its diagnostic.
+    if outcome.status == PublicationStatus::Complete
+        || outcome.evidence.kind == "validation-rejected"
+    {
+        return outcome;
+    }
+    match observe() {
         Ok(head) if head.as_ref() == Some(&pending.planned_head) => {
             Outcome::new(PublicationStatus::Complete, "ref-converged", None, head)
         }