feat(bench): price refusal rendering by strategy and explicit residency - #904
Draft
wenzowski wants to merge 4 commits into
Draft
feat(bench): price refusal rendering by strategy and explicit residency#904wenzowski wants to merge 4 commits into
wenzowski wants to merge 4 commits into
Conversation
`hook::deny_text` has always taken a `first_sighting: bool` and rendered differently on either side of it, and nothing had ever priced the difference. CLOUD-1053 and CLOUD-417 both argue about refusal repetition without a number; this supplies the number and nothing else. `perf` gains the projection — `Strategy x Residency -> first_sighting` — the record, and the report's bytes. The renderers are untouched: `deny_text`, `Refusal::line` and `verdict::render_line` stay the authorities, and a bench that re-implemented any of them would measure itself. TWO CLASSES, ONE PER ROUTE KIND, and that is the half a single-class matrix would have got wrong. A first sighting appends `command` routes only, so `branch write unsafe` (two command routes) prices the renderer's best case while `tool run loose` (one `document` route, no command route) renders the identical bare line cold and warm. THE ROW'S FOURTH EXPECTED RELATIONSHIP IS REFUTED AS STATED, and the tier records the correction rather than the expectation. Over what this repository EMITS, a full warm delivery is not longer than a compact repeat: the carried line for `branch write unsafe` measures 156 characters and 40 estimated tokens, the committed `[refusal] max_tokens` is 24, and `deny_text` drops the routes rather than exceed the budget. So no first sighting in this repository carries a route at all — for that class because the ceiling withholds it, and for the document-only classes because the renderer filters them. The margin is real in the renderer and withheld by the budget, which is why every arm also carries an unbounded column and why those are two cases rather than one: a single `>` over the emitted column would have reported a budget decision as a renderer defect. THE REPORT IS EMITTED IN THE FORMATTER'S OWN SHAPE, which is what keeps one file under one authority. prettier owns every tracked Markdown file including this generated one, so a generator emitting unpadded tables would leave a tree the fixer rewrites and a drift check that then fails against the bytes the fixer just produced. `markdown_table` emits what prettier would leave, so both agree and neither has to yield. The tier carries the drift check rather than the task: the six records are re-rendered in-process and diffed against the committed report, so the file cannot go stale without `test:cargo` reddening — which is what lets the benchmark task stay off the landing path. `engine-perf` joins `MUTANT_GATES` because the module now declares a row; a module that declares one without being in the set censuses as `uncovered`. `address-transport-bench` joins `acquisition_metric`'s table while it is open: that task's own comment claims the assertion covers it and the table did not. Refs: CLOUD-1606
|
Warning Review limit reachedNext included review available in 25 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (5)
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (7)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📝 WalkthroughWalkthroughAdds a deterministic refusal-rendering benchmark for strategy and residency combinations. The benchmark measures emitted and unbounded character and token counts through the shipped refusal path. A new example writes a versioned Markdown report. Merge Risk: ⚪ Minimal · up to This change adds deterministic refusal-render measurements, a generated report, and task/test wiring without changing refusal behavior. The committed report is checked against current rendering, and no merge-blocking risk is evident. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 79.17% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 24 functions across 5 files. (2 skipped: 2 unsupported.) ✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…roken harness The staged tree persists between runs by design — that is what keeps an unchanged source's timestamp and a compiled tier affordable — so `add -A` stages nothing on every run after the first and a plain `git commit` exits 1 with "nothing to commit, working tree clean". `make_a_repository` bailed on that, and the message it bailed with named the wrong thing entirely: it reports a tree that cannot be made a repository, over a tree that already is one, at exit 3. Measured here, and it is why this is a fix rather than a row: the first sweep in this checkout ran, and every sweep after it was could-not-look. That is the shape a reader debugs for an hour before suspecting the harness rather than their own gate — and it made CLOUD-1606's declared mutation unrunnable. `--allow-empty` is the whole change; the case runs the toy sweep TWICE over one root, because a fresh root per run is precisely what hides this. AND THE MUTATION IT UNBLOCKED THEN SURVIVED, which is the finding that came with it. `a_compact_warm_repeat_is_exactly_the_refusal_line` asserted over the EMITTED line, and the emitted line is `Refusal::line()` on both sides of `first_sighting` in this repository: the ceiling withholds the routes for `branch write unsafe` and the renderer appends none for `tool run loose`. So a projection defect is invisible in what this repository emits — a fact about the repository, not a reason to assert less. The case now also asserts the unbounded rendering, which is where `first_sighting` is observable at all, and `mutate sweep` reports the row caught. Refs: CLOUD-1606
…ment The gate's own walk of `crates/batten/tests/**/*.rs` reads 231 against a declared 220, eleven past a tolerance of 10. `main` carries ten of those and this branch one, so this is the sixth entry of the shape the five above it describe: the basis trailing the tree, caught. Neither floor moves, for the 2026-09-06 entry's reason unchanged — a tracked test file stopped being a proxy for a linked stem at the target consolidation, so this basis is a trend counter over a quantity that no longer drives the bytes the floors budget, and an exact floor needs a from-empty build this container cannot hold. Neither `measured` moves either, which is where this parts from the 2026-09-06 entry and follows the 2026-09-05 one. `measured` records when the FLOOR was measured; bumping it to satisfy the refusal's own sentence performs the staleness the field exists to record. A basis at 231 beside floors dated 2026-09-06 says what happened. A matching date would have claimed a measurement nobody took. Measured on this lap, and it is the sixth time the caller has misreported it: `verify` rendered the refusal as "not enough disk to run the gate" with 19154 MB free against a 9472 MB warm floor. Twice the floor, and a stem count is what refused. Refs: CLOUD-1606 Admits: 2f89b34a8d87d2877f624a48fb26743613eab30d430a74515fe063ca5ec3083d Admits-rule: protected-mutation Admits-verdict: path write refused Admits-subject: batten.toml Admits-anchor: call:24c93bf070024ffba2eaeab063321a512188ce62 Admits-epoch: 981bf458ab45817a1c791dfae4d94fcfc212648592a708dc945e59431350169d Admits-author: alec@wenzowski.com Admits-prev: - Admits-answer-lost: Every lap in this checkout stays refused. `target-prune` gates `verify`, so with the basis eleven past its tolerance no receipt can be written and nothing can be readied or landed — by a gate that is, on its own terms, correct: the basis really has trailed the tree, and only the committed config can say so. Admits-answer-precondition: No surface writes this value. `batten target prune` READS `[prune.warm.basis] count` and refuses against it; there is no verb that moves it, and the refusal's own remedy is to move `count` and `measured` in the committed config by hand. The write is two integers, 220 -> 231, and it lands in the diff of the PR closing CLOUD-1606 where a reviewer sees it beside the entry explaining it. Admits-answer-rejected-route: `config read first` points at `batten.toml` and is a READ route; it is how I found the declared 220 and the tolerance, and it cannot change either. `patch run first` is `git restore`, which would discard the basis move and restore the refusal it repairs. Neither reaches the value, which lives only in the committed config.
…s halves Three workspace lints over the new code, and one of them is answered with an `expect` rather than a rewrite because the rewrite would cost the thing the code is for. `match_same_arms` sees `Current`/`Warm` and `FirstFullThenCompact`/`Warm` agreeing today and offers to collapse them: each is one declared row of the strategy table, the first is the anchor a declared mutation rewrites, and a collapsed pair is beyond both the sweep's reach and a reader's. The `expect` says so on the line it describes. The other two are ordinary. `format_push_string` goes by building the pipe line from pieces instead of formatting into a push, and `too_many_lines` by splitting the report's prose preamble from its tables — two halves different people read. `bench/refusal-render/RESULTS.md` regenerates byte-identically across the split, which is what says the refactor moved no measurement. Refs: CLOUD-1606
|
❌ The last analysis has failed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes CLOUD-1606.
What this measures
Six deterministic renderings — three strategies × two explicitly supplied residency inputs — over two committed classes, one per route kind, all through the shipped
hook::deny_textpath.ColdandWarmare benchmark inputs: nothing in this repository detects residency, and a successor that wants to act on it must name its own explicit authority.bench/refusal-render/RESULTS.mdis the record. The drift check is the tier's, not the task's: the six records are re-rendered in-process and diffed against the committed bytes, so the report cannot go stale withouttest:cargoreddening — which is what lets the task stay off the landing path.The finding, and it corrects the row
CLOUD-1606's fourth expected relationship — a full warm delivery is longer than a compact repeat — is false over what this repository emits, and the tier records the correction rather than the expectation.
branch write unsafe(2 command routes)tool run loose(1 document route)No first sighting in this repository carries a route at all, and the two classes reach that by different mechanisms:
branch write unsafedeclares two command routes and would render them, but the carried line is 40 estimated tokens against a committed[refusal] max_tokensof 24, sodeny_textfalls back to the compact form. A budget decision.tool run loosedeclares adocumentroute and no command route, and a first sighting appends command routes only — so it renders the identical line however much budget there is. A renderer gap, which is CLOUD-1637's.Every arm therefore carries an unbounded column beside the emitted one, and those are two test cases rather than one: a single
>over the emitted column would have reported the budget decision as a renderer defect.Notes
//MUTANT current-warm-first-sightinglives on the projection inperf.rs;engine-perfjoinsMUTANT_GATES, since a module that declares a row without being in the set censuses asuncovered.address-transport-benchjoinsacquisition_metric'sBENCH_TASKSwhile that table is open: the task's own comment claims the assertion covers it and the table did not.hook.rs,refusal.rs,verdict.rs.Generated by Claude Code