Skip to content

Replay: ask the config parser before re-deriving an agent exec entry (#453, #521) - #66

Merged
opencdlee-dotcom merged 1 commit into
agent/fable-precision/assemblyfrom
agent/precision-p2a/harness-2
Sep 24, 2026
Merged

opencdlee-dotcom merged 1 commit into
agent/fable-precision/assemblyfrom
agent/precision-p2a/harness-2

Conversation

@opencdlee-dotcom

Copy link
Copy Markdown
Owner

Precision plan, pass 2, step P2a-2: the replay harness now uses the config parser for agent exec entries. Base: agent/fable-precision/assembly.

The defect

_reobserve_agent_surface_answer rebuilt a target or materialized exec entry directly from the recorded fingerprint and the recorded program, then passed it to diff_agent_surface. The config parser never ran, and P2d's fixes live in that parser:

  • Codex's runtime process table no longer registers an exec (#521).
  • A guarded hook line now resolves to its script, not the glue command in front of it (#453: /bin/mkdir became aikit's hook_postwrite.py).

The replay still counted both incidents as re-derived and re-opening.

The fix (harness only)

For target and materialized records, the harness now reads the config at path through snapshot_agent_surface, limited to that one file (_agent_config_files overridden, memoized per path). The parse, the process-table refusal and the resolver are therefore the ones a scan runs.

today's parse outcome
entry no longer registered dropped: "the current parser registers no such entry"
resolves to a different file re-derived against that file: its bytes and custody, matched on the fingerprint the sensor mints for them. If it resolves to nothing with content: dropped
entry and target unchanged as before: held to the recorded bytes, moved-on if they changed
config gone gone
config unreadable not re-derivable
pre-identity positional exec key not re-derivable, never read as a missing entry

A dropped answer can now carry the sensor's reason, counted in stats["dropped_why"] and printed under the not-re-derivable line. Dropped findings still count as re-observed and no longer emitted, so the accounting, the split and the self-check are unchanged.

Live replay (backtest replay --days 30 --reobserve)

  • Before, measured on this base: noise re-opened: 59 of 214 — re-derived 9 (the target), as recorded 50. Re-derived: #352 #382 #406 #413 #414 #432 #452 #453 #521.
  • After: noise re-opened: 57 of 214 — re-derived 7 (the target), as recorded 50. Re-derived: #352 #382 #406 #413 #414 #432 #452.
  • #453: all 424 findings now re-derive against hook_postwrite.py, going from HIGH/- to MEDIUM/local-commit. #521: dropped by the current parser.
  • The self-check passes: re-derived 7 + as recorded 50 = 57 re-opened.

Tests

tests/test_backtest_replay_persistence.py: the agent exec-target class now writes a real config that the parser reads, under the sandbox. It adds five cases: dropped (process table), re-resolved, unchanged-but-moved-on, config gone, and positional key.

Pre-existing red on this base, fixed here: test_a_target_now_proven_committed_is_demoted fails on agent/fable-precision/assembly as it stands (1 != 0 on custody_changed). P2d asks custody at the target's realpath, and the stub compared the temp path as written. On macOS those differ, because the temp dir sits under /var -> /private/var. The stub now compares real paths.

Full suite at 0594c36, python3 -m pytest tests/ -q -p no:cacheprovider: 2417 passed, 7 skipped, 30 xfailed, 41 subtests passed in 1160.08s. The tree sha was 2e56a655 both before and after the run, ~/.aegis/custody.jsonl was byte-identical before and after, and S7's conftest guard is in the tree.

🤖 Generated with Claude Code

…never saw the parser that stopped registering them

_reobserve_agent_surface_answer rebuilt a `target` / `materialized` exec
entry straight from the recorded fingerprint and the recorded program, and
fed it to diff_agent_surface. The config parser was never asked, and P2d's
fixes live there: Codex's runtime process table no longer registers an exec
(#521), and a guarded hook line resolves to its script instead of the glue
command in front of it (#453: /bin/mkdir -> aikit's hook_postwrite.py). The
replay still listed both as re-derived and re-opening.

For those two shapes the config at `path` is now read first by
snapshot_agent_surface over that one file (`_agent_config_files`
overridden, memoized per path), so the parse, the process-table refusal and
the resolver are the ones a scan runs:

- the entry is no longer registered -> dropped, "the current parser
  registers no such entry";
- it resolves to another file today -> re-derived against THAT file's bytes
  and custody, and matched on the fingerprint the sensor mints for them;
  resolved to nothing with content -> dropped;
- entry and target unchanged -> exactly as before (held to the recorded
  bytes, moved-on when they changed);
- config gone -> gone; unreadable -> not re-derivable; a pre-identity
  positional exec key -> not re-derivable, never read as a missing entry.

A dropped answer may now carry the sensor's reason: _reobserve_apply counts
it in stats["dropped_why"] (still reobserved + no_longer_emitted, so the
accounting and the split are unchanged), and the report prints it under the
not-re-derivable line.

Live store: noise re-opened 59 of 214 (re-derived 9) -> 57 of 214, re-derived
7 (the target), as recorded 50; #453's 424 findings re-derive HIGH/- ->
MEDIUM/local-commit on hook_postwrite.py and #521 is dropped.

tests/test_backtest_replay_persistence.py: the agent exec-target class now
writes a real config the parser reads, and adds the dropped, re-resolved,
unchanged-but-moved-on, gone and positional-key cases. Its existing
committed-target test failed on this base before this change: P2d asks
custody at the target's realpath and the stub compared the temp path as
written; it now compares real paths.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@opencdlee-dotcom
opencdlee-dotcom merged commit 3be3e27 into agent/fable-precision/assembly Sep 24, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant