Make the V5 diagnostic harness executable and fail closed - #3603
Merged
Conversation
This was referenced Aug 29, 2026
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3603 +/- ##
=======================================
Coverage 91.85% 91.85%
=======================================
Files 20 20
Lines 6093 6093
=======================================
Hits 5597 5597
Misses 496 496 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b1b6f491c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
August 29, 2026 11:11
8b1b6f4 to
245af42
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 1, 2026 22:15
245af42 to
2ddc311
Compare
jswrenn
approved these changes
Sep 2, 2026
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 2, 2026 20:45
2ddc311 to
98c9f96
Compare
This was referenced Sep 2, 2026
joshlf
changed the base branch from
Guzo775xby4g7ptvtc325vbwh3zf5wxhv
to
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
September 2, 2026 20:45
This was referenced Sep 2, 2026
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
from
September 3, 2026 02:26
7d4ace9 to
5ba5deb
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 03:03
df52065 to
c941428
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
from
September 3, 2026 03:03
5ba5deb to
0393e94
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 04:06
c941428 to
afc8a38
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
2 times, most recently
from
September 3, 2026 04:42
cbcb1ae to
ead3f9e
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
2 times, most recently
from
September 3, 2026 05:14
471908d to
b594f16
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
from
September 3, 2026 05:14
ead3f9e to
122421a
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 05:47
b594f16 to
7617075
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
from
September 3, 2026 05:47
122421a to
13afe9a
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 06:17
7617075 to
365d702
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
from
September 3, 2026 06:17
13afe9a to
6225616
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 06:47
365d702 to
f3b2c36
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
2 times, most recently
from
September 3, 2026 07:16
abd3652 to
e1d8af7
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
2 times, most recently
from
September 3, 2026 07:46
73d3ccd to
b83f623
Compare
joshlf
force-pushed
the
Ghh32fbkyuqkrndyfjzwwarkb4dmfosug
branch
2 times, most recently
from
September 3, 2026 08:17
8417a47 to
a7be91c
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 08:17
b83f623 to
241013e
Compare
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 08:49
241013e to
6f42990
Compare
The first V5 draft described the intended diagnostic study but could not safely execute it. Review found that READY promotion was impossible, several CLI routes had stale arities, padded report IDs disagreed with their validators, host-specific paths destroyed prompt equality, evaluator packets named only digests rather than readable evidence, materiality had no runnable lifecycle, and DRAFT/READY schemas and runtime-state rules contradicted one another. Replace that draft boundary with an authenticated prepare-snapshot, private-review, and finalize lifecycle. The production lock now binds the trusted source declaration, both skill packages, every target, the harness programs, the staged word counter, 120 report prompts/plans/launches, 43 evaluator assignments, hook-specific review contracts and receipts, empty pre-lock runtime state, and a separately custodied external commitment. Synthetic paths carry an authenticated test-only kind and cannot mint production artifacts. Adversarial review exposed further trust failures: executing an unverified candidate verifier, source-copy TOCTOU, ambiguous line-oriented manifests, arbitrary review claims and evaluator launches, stale receipts, unreviewed runtime state, irrelevant schema leakage, optional or crash-unsafe external commitments, and a coherent attack that rebound an F report to target E and a V5 condition to the V4 package. Use injective framed commitments, trusted in-process regeneration, exact artifact/check/evidence inventories, private review copies, atomic no-replace publication, explicit custody-bound recovery, and exact target/condition/package joins to close those failures. Preserve each attack as a negative self-test. Complete the execution protocol around those locked inputs: readable content-addressed packets, two independent consistency reviews, conditional adjudication, materiality review and ledger reconstruction, exact projection and control joins, deterministic aggregate rebuilding, production state authentication, canonical path checks, leases and seals with crash recovery, and fail-closed bound gate evaluation. Unbound caller data cannot make D-STATIC pass. Validation covers prepare, integration, protocol, and draft-verification self-tests; hostile temporary paths; all CLI help surfaces; all JSON parsing and schemas; adversarial provenance, packet, lease, gate, commitment, and assignment mutations; whitespace; and cache hygiene. This remains diagnostic infrastructure: G-ISOLATION and G-OUTPUT-FINALIZATION deliberately remain FAIL, so the commit cannot support a release or terminal-VN claim. gherrit-pr-id: Ghchu3g3fkri2ofhgm5addjto4rfkyvyi Agent-Authored-By: AI agent acting on Josh Liebow-Feeser's behalf
joshlf
force-pushed
the
Ghchu3g3fkri2ofhgm5addjto4rfkyvyi
branch
from
September 3, 2026 11:37
6f42990 to
886e72a
Compare
joshlf
enabled auto-merge
September 3, 2026 11:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first V5 draft described the intended diagnostic study but could not safely execute it. Review found that READY promotion was impossible, several CLI routes had stale arities, padded report IDs disagreed with their validators, host-specific paths destroyed prompt equality, evaluator packets named only digests rather than readable evidence, materiality had no runnable lifecycle, and DRAFT/READY schemas and runtime-state rules contradicted one another.
Replace that draft boundary with an authenticated prepare-snapshot, private-review, and finalize lifecycle. The production lock now binds the trusted source declaration, both skill packages, every target, the harness programs, the staged word counter, 120 report prompts/plans/launches, 43 evaluator assignments, hook-specific review contracts and receipts, empty pre-lock runtime state, and a separately custodied external commitment. Synthetic paths carry an authenticated test-only kind and cannot mint production artifacts.
Adversarial review exposed further trust failures: executing an unverified candidate verifier, source-copy TOCTOU, ambiguous line-oriented manifests, arbitrary review claims and evaluator launches, stale receipts, unreviewed runtime state, irrelevant schema leakage, optional or crash-unsafe external commitments, and a coherent attack that rebound an F report to target E and a V5 condition to the V4 package. Use injective framed commitments, trusted in-process regeneration, exact artifact/check/evidence inventories, private review copies, atomic no-replace publication, explicit custody-bound recovery, and exact target/condition/package joins to close those failures. Preserve each attack as a negative self-test.
Complete the execution protocol around those locked inputs: readable content-addressed packets, two independent consistency reviews, conditional adjudication, materiality review and ledger reconstruction, exact projection and control joins, deterministic aggregate rebuilding, production state authentication, canonical path checks, leases and seals with crash recovery, and fail-closed bound gate evaluation. Unbound caller data cannot make D-STATIC pass.
Validation covers prepare, integration, protocol, and draft-verification self-tests; hostile temporary paths; all CLI help surfaces; all JSON parsing and schemas; adversarial provenance, packet, lease, gate, commitment, and assignment mutations; whitespace; and cache hygiene. This remains diagnostic infrastructure: G-ISOLATION and G-OUTPUT-FINALIZATION deliberately remain FAIL, so the commit cannot support a release or terminal-VN claim.
Agent-Authored-By: AI agent acting on Josh Liebow-Feeser's behalf
This PR is on branch codex/unsafe-rust-stack.
Latest Update: v27 — Compare vs v26
📚 Full Patch History
Links show the diff between the row version and the column version.
⬇️ Download this PR
Branch
git fetch origin refs/heads/Ghchu3g3fkri2ofhgm5addjto4rfkyvyi && git checkout -b pr-Ghchu3g3fkri2ofhgm5addjto4rfkyvyi FETCH_HEADCheckout
git fetch origin refs/heads/Ghchu3g3fkri2ofhgm5addjto4rfkyvyi && git checkout FETCH_HEADCherry Pick
git fetch origin refs/heads/Ghchu3g3fkri2ofhgm5addjto4rfkyvyi && git cherry-pick FETCH_HEADPull
Stacked PRs enabled by GHerrit.