feat: add LS Build Week stale-approval trust gate - #895
Conversation
📝 WalkthroughWalkthroughAdds a deterministic, dependency-free trust gate that validates reviewer identity, review freshness, commit binding, and required CI lanes. It includes four demo fixtures, a trust policy, CLI reports, unit tests, README documentation, and a Build Week submission plan. ChangesTrust Gate Demo
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CLI as Trust-gate CLI
participant Policy as Trust policy
participant Fixture as Demo fixture
participant Gate as evaluate
participant Report as Machine or human report
CLI->>Policy: Load trusted reviewer and required lanes
CLI->>Fixture: Load review and CI evidence
CLI->>Gate: Validate and evaluate evidence
Gate->>Report: Return verdict, reason code, checks, and digests
Report-->>CLI: Print or write deterministic output
Possibly related issues
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
LS multi-model PR review
Model executions
Confirmed findingsNo finding reached independent two-model confirmation. Candidate findingsNo structured candidate finding was produced. Incomplete lanes
Authority boundaryThis output is evidence for human review. It cannot approve or merge the PR, and a single-model finding remains a candidate rather than a gate decision. |
|
@coderabbitai review Please perform a fresh review of exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== PR info ==="
gh pr view "$pr" --repo "$repo" --json title,body,headRefOid,baseRefOid,state,isDraft,files,additions,deletions
echo "=== Commits ==="
gh pr view "$pr" --repo "$repo" --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline}'
echo "=== Files changed ==="
gh pr diff "$pr" --repo "$repo" --name-onlyLength of output: 2948 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== build-week/README.md ==="
gh pr view "$pr" --repo "$repo" --json headRefOid --jq .headRefOid
git clone --depth 50 --branch $(gh pr view $pr --repo $repo --json headRefName --jq .headRefName) https://github.com/$repo.git repo_clone 2>&1 | tail -5
cd repo_clone
git checkout dc2407ef9b1dd2fe9183e8c0482226ee94352b27
echo "=== files ==="
for f in build-week/README.md build-week/demo/stale-approval.json build-week/demo/trusted-current-head.json build-week/policy/trust-policy.json tests/test_build_week_trust_gate.py tools/build_week_trust_gate.py; do
echo "----- $f -----"
cat "$f"
echo
doneLength of output: 30395 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== root listing ==="
ls -la
echo "=== tools dir ==="
ls -la tools
echo "=== tests dir ==="
ls -la tests
echo "=== check for __init__.py or pytest config ==="
fd -HI '(setup.cfg|pyproject.toml|pytest.ini|conftest.py)$' .
echo "=== grep for build_week reference elsewhere ==="
rg -n "build_week" --glob '!build-week/**' --glob '!tools/**' --glob '!tests/**' .Length of output: 22399 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== conftest.py ==="
cat conftest.py
echo "=== pyproject.toml relevant sections ==="
sed -n '1,60p' pyproject.toml
echo "=== check tools/__init__.py existence ==="
fd -HI '__init__.py$' tools
echo "=== check other tools test importing tools package similarly ==="
rg -n "^from tools\." tools/test_*.py | head -20Length of output: 4283 🏁 Script executed: #!/bin/bash
set -euo pipefail
gh issue view 897 --repo safal207/LS --json title,state,url 2>&1 || echo "issue 897 not found or inaccessible"Length of output: 404
Scope verification — all requested points confirmed:
Cross-checked Two non-blocking nits for a future follow-up:
Nice, clean fail-closed design for a first slice — no bugs found that would affect the stale-approval vs. trusted-head demo behavior. ✅ Action performedReview finished.
|
|
@coderabbitai review Please perform a fresh review of exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== PR info ==="
gh pr view "$pr" --repo "$repo" --json headRefOid,baseRefOid,state,isDraft
echo "=== Commits ==="
gh pr view "$pr" --repo "$repo" --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline}'
git clone --depth 100 --branch $(gh pr view $pr --repo $repo --json headRefName --jq .headRefName) https://github.com/$repo.git repo_clone 2>&1 | tail -5
cd repo_clone
git checkout a4236da2120f52f0d35ce27c1bc40fa5b698531c
echo "=== files changed since dc2407e ==="
git diff --name-status dc2407ef9b1dd2fe9183e8c0482226ee94352b27..a4236da2120f52f0d35ce27c1bc40fa5b698531c
echo "=== full diff ==="
git diff dc2407ef9b1dd2fe9183e8c0482226ee94352b27..a4236da2120f52f0d35ce27c1bc40fa5b698531cLength of output: 10930 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== full tools/build_week_trust_gate.py ==="
cat -n tools/build_week_trust_gate.py
echo "=== run tests ==="
python3 -m unittest -v tests/test_build_week_trust_gate.py 2>&1 || true
echo "=== run fixtures manually ==="
for f in build-week/demo/*.json; do
echo "--- $f ---"
python3 tools/build_week_trust_gate.py "$f" --verify-expected --format json 2>&1 | tail -20
doneLength of output: 30656
Focus areas — all confirmed:
Ran the full suite and all four fixtures locally:
Both nits from the previous review are now resolved with dedicated test coverage. No new issues found in this slice — the two adversarial fixtures are deterministic, correctly wired into the trusted-reviewer policy path (not the fixture's own claims), and preserve the three-state PASS/FAIL/NOT_RUN vocabulary end-to-end. ✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@BUILD_WEEK_2026_PLAN.md`:
- Around line 3-5: Update BUILD_WEEK_2026_PLAN.md to remove provisional and
unpublished-rule placeholders across the status, submission assets, timeline,
and next actions sections. Replace them with the published Devpost rules,
submission requirements, judging criteria, and corresponding dates, while
preserving the plan’s existing structure.
In `@tools/build_week_trust_gate.py`:
- Around line 491-492: Update the report output handling around
args.report_out.write_text to catch OSError, emit a concise diagnostic, and
return exit code 2 through the CLI’s controlled input-error path. Preserve the
existing successful write behavior and JSON output generation.
- Around line 364-383: Update the lane classification logic around the status
and head_sha checks so the current-head comparison runs first. When
lane["head_sha"] differs from current_head, always emit STALE_REQUIRED_LANE,
including when lane["status"] is "FAIL"; only classify matching-head failures as
REQUIRED_LANE_FAILED.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e8ace999-e339-4c96-9b51-a61fa8cabc60
📒 Files selected for processing (9)
BUILD_WEEK_2026_PLAN.mdbuild-week/README.mdbuild-week/demo/required-check-not-run.jsonbuild-week/demo/spoofed-reviewer.jsonbuild-week/demo/stale-approval.jsonbuild-week/demo/trusted-current-head.jsonbuild-week/policy/trust-policy.jsontests/test_build_week_trust_gate.pytools/build_week_trust_gate.py
|
@coderabbitai review Please review exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== PR info ==="
gh pr view "$pr" --repo "$repo" --json headRefOid,baseRefOid,state,isDraft
echo "=== Commits ==="
gh pr view "$pr" --repo "$repo" --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline}'
git clone --depth 100 --branch $(gh pr view $pr --repo $repo --json headRefName --jq .headRefName) https://github.com/$repo.git repo_clone 2>&1 | tail -5
cd repo_clone
echo "=== trying to checkout requested sha ==="
git checkout 0d4021a769b5c7e5c63c0241d2234eef2b2e537e 2>&1 || echo "CHECKOUT FAILED"
echo "=== log ==="
git log --oneline -15Length of output: 3149 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== diff since a4236da ==="
git diff --name-status a4236da2120f52f0d35ce27c1bc40fa5b698531c..0d4021a769b5c7e5c63c0241d2234eef2b2e537e
echo "=== full diff ==="
git diff a4236da2120f52f0d35ce27c1bc40fa5b698531c..0d4021a769b5c7e5c63c0241d2234eef2b2e537eLength of output: 11722 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== full file for context ==="
sed -n '1,40p' tools/build_week_trust_gate.py
echo "..."
sed -n '330,420p' tools/build_week_trust_gate.py
echo "=== run tests ==="
python3 -m unittest -v tests.test_build_week_trust_gate 2>&1 || true
echo "=== run fixtures manually ==="
for f in build-week/demo/*.json; do
echo "--- $f ---"
python3 tools/build_week_trust_gate.py "$f" --verify-expected --format json 2>&1 | tail -20
echo "exit: $?"
done
echo "=== check argparse for report-out and main structure ==="
sed -n '440,510p' tools/build_week_trust_gate.pyLength of output: 14322
All three repaired areas confirmed:
Local validation reproduced:
No new issues found in this slice. The classification fix in particular closes a real gap between the demo's "stale evidence is inadmissible" thesis and the implementation's prior behavior — good catch and clean fix with dedicated regression coverage. ✅ Action performedReview finished.
|
|
@coderabbitai review Please review exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== PR info ==="
gh pr view "$pr" --repo "$repo" --json headRefOid,baseRefOid,state,isDraft
echo "=== Commits ==="
gh pr view "$pr" --repo "$repo" --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline}'
git clone --depth 100 --branch $(gh pr view $pr --repo $repo --json headRefName --jq .headRefName) https://github.com/$repo.git repo_clone 2>&1 | tail -5
cd repo_clone
echo "=== trying to checkout requested sha ==="
git checkout 299db4b239eddad32b621f31bd8b47de25f40fd7 2>&1 || echo "CHECKOUT FAILED"
echo "=== log ==="
git log --oneline -15Length of output: 3226 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== diff since 0d4021a ==="
git diff --name-status 0d4021a769b5c7e5c63c0241d2234eef2b2e537e..299db4b239eddad32b621f31bd8b47de25f40fd7
echo "=== full diff ==="
git diff 0d4021a769b5c7e5c63c0241d2234eef2b2e537e..299db4b239eddad32b621f31bd8b47de25f40fd7Length of output: 6857 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== main() argparse section ==="
sed -n '1,60p' tools/build_week_trust_gate.py | grep -n "argparse\|add_argument\|policy" || true
rg -n "add_argument|--policy|--verify-expected|--report-out|--format" tools/build_week_trust_gate.py
echo "=== permissions ==="
ls -la scripts/run_build_week_demo.sh
echo "=== run tests ==="
python3 -m unittest -v tests/test_build_week_trust_gate.py tests/test_build_week_demo.py 2>&1 || true
echo "=== run demo script directly from repo root ==="
./scripts/run_build_week_demo.sh; echo "exit: $?"
echo "=== run demo script from another cwd ==="
cd /tmp
"$OLDPWD/scripts/run_build_week_demo.sh" 2>&1 || true
cd -
echo "=== run demo script with absolute path from /tmp ==="
mkdir -p /tmp/elsewhere && cd /tmp/elsewhere
bash "$(pwd)/../../repo_clone/scripts/run_build_week_demo.sh" 2>&1 || true
REPO_ROOT=$(cd - >/dev/null && pwd)
echo "repo root: $REPO_ROOT"
bash "$REPO_ROOT/scripts/run_build_week_demo.sh"; echo "exit: $?"Length of output: 5538 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== git file mode ==="
git ls-files -s scripts/run_build_week_demo.sh
echo "=== check shebang and no external deps besides python3/bash ==="
head -5 scripts/run_build_week_demo.sh
echo "=== check README final content ==="
cat build-week/README.mdLength of output: 3491
All requested focus areas confirmed:
Local validation reproduced exactly:
No issues found in this slice — the one-command demo wrapper is genuinely portable, fails closed on both scenario mismatches and missing interpreters, and is backed by a real subprocess-level regression test rather than just unit-level coverage of the underlying gate. ✅ Action performedReview finished.
|
|
@coderabbitai review Please review exact head |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
pr=895
echo "=== PR info ==="
gh pr view "$pr" --repo "$repo" --json headRefOid,baseRefOid,state,isDraft
echo "=== Commits ==="
gh pr view "$pr" --repo "$repo" --json commits --jq '.commits[] | {oid: .oid, messageHeadline: .messageHeadline}'
git clone --depth 100 --branch $(gh pr view $pr --repo $repo --json headRefName --jq .headRefName) https://github.com/$repo.git repo_clone 2>&1 | tail -5
cd repo_clone
echo "=== trying to checkout requested sha ==="
git checkout 97544c00c83eff70fb10592977173094b92f3d55 2>&1 || echo "CHECKOUT FAILED"
echo "=== log ==="
git log --oneline -15Length of output: 3321 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== diff since 299db4b ==="
git diff --name-status 299db4b239eddad32b621f31bd8b47de25f40fd7..97544c00c83eff70fb10592977173094b92f3d55
echo "=== full diff ==="
git diff 299db4b239eddad32b621f31bd8b47de25f40fd7..97544c00c83eff70fb10592977173094b92f3d55Length of output: 16901 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== checkout evidence subject commit 299db4b for reproducibility check ==="
git checkout 299db4b239eddad32b621f31bd8b47de25f40fd7 -- . 2>&1 || true
git status --short | head
echo "=== run gate against stale-approval fixture and diff vs saved example (excluding schema-neutral formatting) ==="
python3 tools/build_week_trust_gate.py build-week/demo/stale-approval.json --policy build-week/policy/trust-policy.json --format json > /tmp/generated_report.json 2>&1 || true
cat /tmp/generated_report.json
echo "=== diff generated vs stored example ==="
git show 97544c00c83eff70fb10592977173094b92f3d55:build-week/evidence/trust-report.example.json > /tmp/stored_example.json
python3 - <<'EOF'
import json
gen = json.load(open("/tmp/generated_report.json"))
stored = json.load(open("/tmp/stored_example.json"))
gen_keys = set(gen.keys())
stored_keys = set(stored.keys())
print("missing in gen:", stored_keys - gen_keys)
print("extra in gen:", gen_keys - stored_keys)
for k in sorted(stored_keys & gen_keys):
if gen[k] != stored[k]:
print(f"DIFF at {k}:\n gen={gen[k]}\n stored={stored[k]}")
EOFLength of output: 4382 🏁 Script executed: #!/bin/bash
set -euo pipefail
repo="safal207/LS"
echo "=== verify github actions run ids belong to head sha 299db4b ==="
for run_id in 29303554682 29303554670 29303554694 29303554693; do
echo "--- run $run_id ---"
gh api "repos/$repo/actions/runs/$run_id" --jq '{name, head_sha, status, conclusion, event}' 2>&1 || echo "FAILED to fetch $run_id"
done
echo "=== verify the linked review comment exists ==="
gh api "repos/$repo/issues/comments/4965121921" --jq '{user: .user.login, body: .body[0:200], html_url}' 2>&1 || echo "COMMENT NOT FOUND"Length of output: 1913 🏁 Script executed: #!/bin/bash
set -euo pipefail
cd repo_clone
echo "=== search for feedback session id fabrication ==="
rg -ni "session.?id" build-week/evidence/ BUILD_WEEK_2026_PLAN.md build-week/README.mdLength of output: 955 All checks pass, confirming this documentation/evidence commit is honest and internally consistent.
All five focus areas confirmed:
No issues found in this documentation/evidence slice — the snapshot's claims are independently verifiable against live GitHub state and the local tool, not just self-asserted. ✅ Action performedReview finished.
|
safal207
changed the title
1 participant
What changed
Implements LS — Trust Layer for AI Software Delivery for the OpenAI Build Week Developer Tools track.
The focused story is:
This PR also adds an exact-source-SHA Docker clean-room path so a judge can reproduce the demo without installing project dependencies.
Type of change
Why it matters
AI review signals can be stale, spoofed, or incomplete. LS proves whether a specific decision belongs to the current code state and came through an allowed evidence-bearing route before that decision can influence delivery.
Fastest judge path
Prerequisite: Docker Engine or Docker Desktop.
Native alternative:
Expected matrix:
Evidence
caaa5ed299db4b97544c03b5b048Checks
Screenshots / video
Public Build Week demo video: pending. The final recording will show the stale-approval attack, the block, the trusted control, and how Codex + GPT-5.6 were used.
Trust boundary
TRUSTEDdoes not perform or authorize autonomous delivery; explicit human authorization remains required.Status
This PR remains draft while the public video,
/feedbackSession ID, and Devpost submission are completed.Refs #897