Skip to content

Per-run trajectory: a kernel-written, hash-chained event timeline - #573

Merged
renmengye merged 8 commits into
mainfrom
feat/trajectory
Oct 24, 2026
Merged

renmengye merged 8 commits into
mainfrom
feat/trajectory

Conversation

@renmengye

Copy link
Copy Markdown
Member

Step 1 of traceable runs: one file per run, runs/<run-id>/trajectory.jsonl, that records what the kernel did as a hash-chained timeline. The operational files (state.json, launches.jsonl, inbox/outbox, session records) are untouched; the trajectory records the same moments as events.

Format. One JSON object per line, keys sorted: {seq, ts, type, actor, data, refs: [{path, sha256}], prev}. prev is the SHA-256 of the previous line's bytes; the first line chains from the run id. Written only by the kernel, under a lock, in the run directory (outside the session's workspace). Best-effort: a write failure of any kind logs a warning and never fails a tick or session. Capped at 4 MB.

Events. session.start/session.end (end refs the native session log by hash), syscall.decision (verb, accepted/refused, reason), launch.submit/launch.result (metadata only, no job stdout), measure (baseline, candidate, credited; once per real measurement; self-reported claims marked), inbox.deliver (seq range, kinds, origins, no bodies), wake, pr.open, ending (refs the report).

Reading it. outerloop trajectory <run-id> [--verify] [--json]: a timeline, and with --verify the chain check (sequence gaps, changed or missing lines). Design note: docs/design/trajectory.md.

Compatibility. No state format changes: a new file only. Runs started under an older kernel have no trajectory and read as "no trajectory"; an older kernel ignores the file. Legacy fixture: a run directory without the file (tests/test_trajectory.py). Upgrading: line in the CHANGELOG.

Later steps (not here): tool-call events extracted from native logs, measurement provenance (image digest, artifact hashes, the chain head in the ledger row), a traceable-numbers check, attestation.

Deferred from review: an append reads the whole file under the lock (bounded by the 4 MB cap); the session-end ref rehashes a native log whose digest the evidence record already has.

Built by codex; reviewed by codex and Claude; review fixes by codex.

…thout stdout, one measure per measurement, explicit decision verbs, self-report measures
@renmengye renmengye added the outerloop:review re-request the advisory review label Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — reviewed head abb88dd2 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: nothing blocking — 1 advisory note.

1 finding attached to the lines below.

The trajectory command treats unreadable trajectory files as missing histories unless verification is requested.

Comment thread src/outerloop/trajectory.py Resolved

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2 (re-run on the same head) — reviewed head abb88dd2 — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 1 blocking, 2 advisory.

2 findings attached to the lines below.

Advisory (non-blocking):

  • Local wakes can be recorded after the ending event. [lifecycle] _wake runs local dispatch synchronously, so it can call finish_run and append ending before sweep appends wake after _wake returns, reversing the event order in the timeline. (src/outerloop/tick.py:619; high confidence)

Merged verdict: 3 findings (1 blocking): scope-violation refusals are incorrectly logged as measurements; unreadable trajectories are treated as absent; local-mode wake and ending events can be reversed. Rejected: none.

Comment thread src/outerloop/orchestrator.py Resolved
Comment thread src/outerloop/trajectory.py Resolved
@renmengye

Copy link
Copy Markdown
Member Author

Advisory on wake vs ending order in local mode: deferred. Every event carries its timestamp, and a local wake that ends the run inside the same call is the only case; recording the wake before dispatch would log wakes that did not happen.

@renmengye renmengye added outerloop:review re-request the advisory review and removed outerloop:review re-request the advisory review labels Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3 — reviewed head d8f8bbc0 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

@renmengye renmengye added outerloop:review re-request the advisory review and removed outerloop:review re-request the advisory review labels Oct 24, 2026
@renmengye

Copy link
Copy Markdown
Member Author

Live test (one machine, local forge, quickstart, Claude author, this branch at c3576f8 plus the first review fixes): one attempt on denoise, PR opened in about 3 minutes. outerloop trajectory <run> --verify:

0 session.start   {"backend": "claude", "model": "claude-opus-5-5", "session_id": "", "stage": "author"}
1 session.end     {"outcome": "end_turn", "is_error": false, "session_id": "563dc7f2-…", …}
2 syscall.decision {"verb": "submit", "outcome": "accepted"}
3 measure         {"baseline": 0.010071, "candidate": 0.000231, "credited": true, "head_sha": "daacadd…", "metric": "mean_mse"}
4 pr.open         {"head_sha": "daacadd…", "url": "<state root>/forge/me/quickstart.git#pull/1"}
chain intact

One defect found: pr.open (and the other events attempt.py records with at=now) carry the attempt's start time, so event 4's ts is earlier than event 0's. The chain order is right; the timestamp is not. Fixing in a follow-up PR (drop at=now there, record wall-clock time) so this PR does not go past its review cap.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 4 — reviewed head ce15ec46 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 1 blocking, 1 advisory.

1 finding attached to the lines below.

Advisory (non-blocking):

  • Steward-created pull requests are absent from the trajectory. The steward creates a PR at this call and stores its URL, but unlike the climb publish path it never records a pr.open event, leaving that kernel action out of the per-run timeline. (src/outerloop/steward.py:605; high confidence)

The trajectory can retain a stale ending report hash and misses steward PR openings.

Comment thread src/outerloop/attempt.py Resolved
Trajectory follow-up: wall-clock timestamps; ending ref matches the steward's report
@renmengye
renmengye merged commit 4abe343 into main Oct 24, 2026
1 check passed
@renmengye
renmengye deleted the feat/trajectory branch October 24, 2026 12:02
@renmengye renmengye mentioned this pull request Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

outerloop:review re-request the advisory review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant