Skip to content

Protocol friction from the trajectory: refusals per run by author model (#487 part 3) - #577

Merged
renmengye merged 3 commits into
mainfrom
feat/friction-metric
Oct 24, 2026
Merged

renmengye merged 3 commits into
mainfrom
feat/friction-metric

Conversation

@renmengye

Copy link
Copy Markdown
Member

Part 3 of #487: protocol friction as a release metric.

  • trajectory.summarize(events) counts syscall.decision events, refusals by verb, and the distinct backend/model pairs from session.start. There are no new events and no format change.
  • outerloop trajectory <run-id> ends with one line such as refusals: 3 of 4 decisions (submit 2, message 1); author: claude/model-x. It prints nothing extra for a run with no trajectory, and the --json output shapes are unchanged.
  • RELEASING.md: pre-release canary runs report refusals per run by author model; a release whose refusals per run rise explains why in the release PR.
  • The agent-text inventory now includes the operator message note from outerloop message, which the part-1 review flagged as missing.

Gate: ruff check, ruff format --check, mypy, pytest (3192 passed, 7 skipped).

Built by codex (GPT); reviewed by Claude.

@renmengye renmengye added the outerloop:review re-request the advisory review label Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 1 — reviewed head cf4d3215 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 1 blocking, 0 advisory.

1 finding attached to the lines below.

Malformed trajectory event data can crash the new summary.

Comment thread src/outerloop/trajectory.py Resolved

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 2 (re-run on the same head) — reviewed head cf4d3215 — reviewer summarizer:hermes/gpt-5.6-terra over coverage+credentials+deployment+general+lifecycle+prose.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: 2 blocking, 1 advisory.

3 findings attached to the lines below.

Verdict: request changes. Blocking: trajectory summary includes panel judge sessions in its author metric, and malformed non-object event data can crash the command. Suggestion: clarify the changelog sentence with the supplied rewrite. Rejected: none; coverage, credentials, and general were merged as the same malformed-data defect, and deployment and lifecycle were merged as the same panel-author defect.

Comment thread src/outerloop/trajectory.py Resolved
Comment thread src/outerloop/trajectory.py Resolved
Comment thread CHANGELOG.md Resolved
@renmengye renmengye added outerloop:review re-request the advisory review and removed outerloop:review re-request the advisory review labels Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 3 — reviewed head 76598934 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

@renmengye renmengye added outerloop:review re-request the advisory review and removed outerloop:review re-request the advisory review labels Oct 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round 4 — reviewed head fa973384 — reviewer hermes/gpt-5.6-terra.

terra
Advisory findings from outerloop — the code owner decides. Reply to disagree; the outerloop:no-review label opts this PR out.

Verdict: no defects found.

@renmengye
renmengye merged commit fc104cd into main Oct 24, 2026
5 checks passed
@renmengye
renmengye deleted the feat/friction-metric branch October 24, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

outerloop:review re-request the advisory review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant