render-pipeline: measure cross-reference invalidation (step 0 of #575; no behaviour change)

[d-q222]

What

Step 0 of the incremental cross-references proposal (#575): measure before choosing. No behaviour change and no design decision.

• RenderCache::counters() (crates/render-pipeline/src/incremental.rs). Plain Cell hit and miss counters for the adapted and assembled maps, and a label_passes counter that render_cached bumps once per layout pass. They are returned together with the existing blocks stats as CacheCounters, and since() gives deltas. stats() is unchanged, and nothing on the render path reads the counters.
• examples/xref_measure.rs runs compiler: 3x faster large-document compile — shaping memo, glyph-table index, lent cleveref config (#65) #535's 450-section article through protocol::handle_line with a warm cache. The generator is byte-identical to compiler: 3x faster large-document compile — shaping memo, glyph-table index, lent cleveref config (#65) #535's (637 761 B). The bench also runs a +toc+pageref variant. It measures three edits: no-label, renumber (a \section inserted before section 1) and move-label (a paragraph with no label moved across a heading, so that one \label changes page). It reports the misses for each map, the layout passes, and the median-of-5 latency.
• tests/cache_counters.rs checks that a cold render counts misses and a warm one counts hits, and that the pass counter equals Rendered::passes.
• docs/design/incremental-cross-references-measurements.md holds the results and the exact commands. It is a separate file, so it does not conflict with docs: design proposal — incremental cross-references (for Commander decision) #575.

document	edit	adapted miss / lookups	blocks miss / lookups	assembled miss / lookups	layout passes	handle_line ms (median)	render ms (median)
535	no-label	0 / 5013	1 / 5461	1 / 5461	1	18493.1	18463.1
535	renumber	4117 / 5015	902 / 5463	902 / 5463	1	17383.1	17354.2
535	move-label	1 / 5013	3 / 5461	3 / 5461	1	16189.0	16162.4
535+toc+pageref	no-label	0 / 10026	1 / 10924	1 / 5462	2	34935.8	34902.1
535+toc+pageref	renumber	8234 / 10030	1239 / 10928	1236 / 5464	2	35206.4	35172.1
535+toc+pageref	move-label	4116 / 10026	4 / 10924	4 / 5462	2	35405.4	35373.7

The cold render takes 18 278 ms for 535 and 34 738 ms for 535+toc+pageref. The cold adapted count is 4 115 misses out of 5 013 lookups per pass, so every distinct block misses.

Why

#575 §5 step 0 asks whether the IDE worker is already fast on these three edits before any option is chosen. Findings:

1. §2.2's invalidation analysis is confirmed.
   • renumber misses adapted for every distinct block in every pass.
   • move-label misses it for every distinct block in pass 2.
   • Below the adapter, reuse stays selective: 1–4 block misses, or 902–1 239 for renumber.
2. Warm latency equals cold latency on every edit (16–18 s with one pass, about 35 s with two). The cache misses are not what costs the time.
3. The time is in adapter::adapt_cached, and it grows quadratically.
   • stages, run warm: adapt takes 2 013 ms at 150 sections and 16 368 ms at 450. Parse plus typeset plus assemble is under 100 ms.
   • Temporary probes (not committed) put about 95% of the adapt time in split_at_page_breaks. Three per-block walks of the source prefix account for it: in_theorem_environment (717 ms at 150 sections; added by 2945f61a), the ListItem branch (831 ms; list_stack_at and others) and the after-list branch (290 ms).
   • This is recorded as a finding for whoever owns adapter.rs, not as a proposal. Until it is fixed, render-pipeline labels_fp narrowing or page seeding could not show a latency win on this document.

Overlap check

• Open PRs searched for RenderCache: FT-070 memory: measure the breakdown, derive the per-glyph carets (Cluster 160 -> 88 B) #232, render-pipeline: layout cache keys tag each kind by its own discriminant (3 colliding tags, latent) #182, FT-070: a negotiated page window — the fix for a 500 KB document that has no reply #294, FT-070 memory: pages reference the assembled items instead of cloning them (-244 MiB at 2 MB) #283 and FT-070: engine performance harness, committed baseline and a >5% regression gate #206. None adds counters. FT-070 memory: pages reference the assembled items instead of cloning them (-244 MiB at 2 MB) #283 and FT-070: profile the warm keystroke, and stop copying a restored block's item list #273 touch incremental.rs, but this PR only adds fields and methods, so there is no textual overlap in the hunks they change.
• Also searched cross-reference (docs: design proposal — incremental cross-references (for Commander decision) #575, render-pipeline: floats numbered in \include reading order, LoF/LoT and \ref agree (stacked on #537 and #538) #578, compiler: 3x faster large-document compile — shaping memo, glyph-table index, lent cleveref config (#65) #535, compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535) #579), split_at_page_breaks (render-pipeline: thread VSpace stretch/shrink into vspace_flex (GH-VSKIP-GLUE-STRETCH slice 2, needs re-pin past #606) #607, which threads vspace flex, not the scans), in_theorem_environment (none) and list_stack_at (render-pipeline: a verbatim body inside a list is a trivlist inside that list (14/17) #268 and render-pipeline: secnumdepth, description, \baselinestretch and run-in headings #292, both behavioural).
• Claims: only GH-XREF-MEASURE-STEP0 (daniel-parent) matched render-pipeline/xref.

Test results

cargo test --test cache_counters
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s
cargo test --test incremental
test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 375.30s
cargo run --release --example xref_measure -- 5 450
exit=0 (table above; load average 23.30 20.81 19.26 at the start)
generator check against #535's large_doc_bench.rs: identical; 637761 bytes

Not done

• No fix to the adapter's prefix scans: they are out of scope and belong to the owner of adapter.rs. There is no change to labels_fp or to page seeding (that waits on the Commander's ruling on docs: design proposal — incremental cross-references (for Commander decision) #575).
• handle_line latency includes an over-limit failure reply: 17.6 MB is over the 16 MiB line limit, and FT-070: a negotiated page window — the fix for a 500 KB document that has no reply #294 is the fix. render ms is the meaningful figure.
• move-label moves one section heading's page; it cannot move one label in isolation, because a paragraph has to be moved to do it. Exactly 1 of the 450 headings changed page in both documents.

🤖 Generated with Claude Code

https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

[d-q222]

daniel-parent independent review (subagent)
Verdict: APPROVE

1. Code read confirms the counters are genuinely inert. RenderCache::stats() (crates/render-pipeline/src/incremental.rs) is byte-for-byte unchanged — still (*self.hits.borrow(), *self.misses.borrow()), reading only the original hits/misses RefCells, untouched by the new Cell fields. adapted() and assembled() preserve their exact prior return value (let hit = ...; bump(...); hit), so no caller's control flow changes. note_label_pass() is pub(crate), called only from render_cached's loop in crates/render-pipeline/src/lib.rs right after the existing passes += 1; and before the existing adapter::adapt_cached(...) call — the diff there is exactly those three added lines, nothing else in the hunk changed. counters()/CacheCounters/since() are new, additive, read-only, and their only readers are the new example and test. RenderCache still derives Default, so new()'s initialization is untouched.
2. Display-list diff on 20 real fixtures, before (origin/main 36fe7ec3) vs after (PR head dbce9043): toc/{01,05,09,10,15,19,21,24,33}-*.tex, thebibliography/main.tex, floats/08-table-ref.tex, float-body/{01,02,18}-*.tex, display-placement/fixtures/{15-align-tag-notag-eqref,17-numberwithin-section,18-subequations}.tex, footnotes/01-single.tex, multicol/01-two-balanced.tex, math-symbols/03-binary.tex — biased toward cross-reference/label/toc/bibliography content since that's the code path touched. Each fixture was sent as two requests (revision 1 "cold", revision 2 identical text "warm") in one flashtex-render worker session (protocol::serve, one live RenderCache across both, matching the real IDE path) with display-list-v2 negotiated. All 20 fixtures accepted the capability and returned status: "ok" on both requests (confirming adapted()/assembled()/note_label_pass() were actually exercised on both the miss and hit paths, not skipped), and diff -q found 0 byte differences across all 20 × 4 output lines (~50 MB of base-checkout output compared).
3. cargo test --release --test cache_counters on the PR-613 worktree: counters_track_lookups_and_label_passes ... ok.
4. Bonus corroboration beyond the required scope: the pre-existing tests/incremental.rs::incremental_output_is_byte_identical_over_200_edits_of_a_27_page_document also passes on the PR-613 worktree (200 edits, byte-identical), independently reinforcing the no-behavior-change claim.