Repository navigation
Per-run trajectory: a kernel-written, hash-chained event timeline - #573
Conversation
… `outerloop trajectory`
…thout stdout, one measure per measurement, explicit decision verbs, self-report measures
There was a problem hiding this comment.
Round 1 — reviewed head abb88dd2 — reviewer hermes/gpt-5.6-terra.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: nothing blocking — 1 advisory note.
1 finding attached to the lines below.
The trajectory command treats unreadable trajectory files as missing histories unless verification is requested.
There was a problem hiding this comment.
Round 2 (re-run on the same head) — reviewed head abb88dd2 — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 2 advisory.
2 findings attached to the lines below.
Advisory (non-blocking):
- Local wakes can be recorded after the ending event. [lifecycle] _wake runs local dispatch synchronously, so it can call finish_run and append ending before sweep appends wake after _wake returns, reversing the event order in the timeline. (
src/outerloop/tick.py:619; high confidence)
Merged verdict: 3 findings (1 blocking): scope-violation refusals are incorrectly logged as measurements; unreadable trajectories are treated as absent; local-mode wake and ending events can be reversed. Rejected: none.
…dable chain is an error, not absent
|
Advisory on wake vs ending order in local mode: deferred. Every event carries its timestamp, and a local wake that ends the run inside the same call is the only case; recording the wake before dispatch would log wakes that did not happen. |
… eval stays recorded
|
Live test (one machine, local forge, quickstart, Claude author, this branch at c3576f8 plus the first review fixes): one attempt on One defect found: |
There was a problem hiding this comment.
Round 4 — reviewed head ce15ec46 — reviewer hermes/gpt-5.6-terra.
terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.
Verdict: 1 blocking, 1 advisory.
1 finding attached to the lines below.
Advisory (non-blocking):
- Steward-created pull requests are absent from the trajectory. The steward creates a PR at this call and stores its URL, but unlike the climb publish path it never records a pr.open event, leaving that kernel action out of the per-run timeline. (
src/outerloop/steward.py:605; high confidence)
The trajectory can retain a stale ending report hash and misses steward PR openings.
Trajectory follow-up: wall-clock timestamps; ending ref matches the steward's report
# Conflicts: # CHANGELOG.md
Step 1 of traceable runs: one file per run,
runs/<run-id>/trajectory.jsonl, that records what the kernel did as a hash-chained timeline. The operational files (state.json, launches.jsonl, inbox/outbox, session records) are untouched; the trajectory records the same moments as events.Format. One JSON object per line, keys sorted:
{seq, ts, type, actor, data, refs: [{path, sha256}], prev}.previs the SHA-256 of the previous line's bytes; the first line chains from the run id. Written only by the kernel, under a lock, in the run directory (outside the session's workspace). Best-effort: a write failure of any kind logs a warning and never fails a tick or session. Capped at 4 MB.Events.
session.start/session.end(end refs the native session log by hash),syscall.decision(verb, accepted/refused, reason),launch.submit/launch.result(metadata only, no job stdout),measure(baseline, candidate, credited; once per real measurement; self-reported claims marked),inbox.deliver(seq range, kinds, origins, no bodies),wake,pr.open,ending(refs the report).Reading it.
outerloop trajectory <run-id> [--verify] [--json]: a timeline, and with--verifythe chain check (sequence gaps, changed or missing lines). Design note:docs/design/trajectory.md.Compatibility. No state format changes: a new file only. Runs started under an older kernel have no trajectory and read as "no trajectory"; an older kernel ignores the file. Legacy fixture: a run directory without the file (
tests/test_trajectory.py).Upgrading:line in the CHANGELOG.Later steps (not here): tool-call events extracted from native logs, measurement provenance (image digest, artifact hashes, the chain head in the ledger row), a traceable-numbers check, attestation.
Deferred from review: an append reads the whole file under the lock (bounded by the 4 MB cap); the session-end ref rehashes a native log whose digest the evidence record already has.
Built by codex; reviewed by codex and Claude; review fixes by codex.