compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535) by d-q222 · Pull Request #579 · flash-tex/flashtex

d-q222 · 2026-09-15T08:13:20Z

Stacked on #535 (base of the comparison: #535 head 3c2b0e11). Task GH-COMPILER-PERF-2, issue #65. Merge #535 first; this branch contains its commits.

What

The next hotspots from #535's profile, fixed without changing output. Only crates/compiler/src is touched (layout.rs, incremental.rs, protocol.rs, parser.rs); no bench or tool changes.

Placed-item copies (layout.rs). render_prepared_block copied every placed TextItem (its String included) into a fragment Vec. Every caller except incremental reuse threw it away: layout, layout_with_constraints, each cross-reference convergence pass in layout_converged_with_options, and the \tableofcontents heading. Those callers now use a new private render_block. It does the same layout without collecting. render_prepared_block is render_block plus the collection, so the incremental path is unchanged. This was ~6–10% of samples.
Session output clone (incremental.rs, protocol.rs). The new Session::compile_project_borrowed returns (&CompileOutput, ReuseStats). The output is moved into the retained Revision rather than cloned into it. compile_project_with keeps its signature and clones from the retained copy, so it makes one deep copy as before. The runtime-v1 transport (protocol::compile) only reads the pages, so it now borrows them and clones just the diagnostics it extends. The IDE path no longer deep-copies every block and placed item per keystroke.
Shaping-memo hashing (layout.rs). A memo hit hashed the word twice (contains_key, then index). It now uses one get, and a miss inserts after reading. No FxHash-style hasher is in flashtex-compiler's dependency tree (rustc-hash only appears in other crates' lockfiles), so none was added. The remaining SipHash cost is ~3.5% of samples, plus ~1.8% key memcmp.

Two small fixes in the same paths, found by the profile:

protocol::bound_pages serialised every page to measure it, cloned every kept page, and then serialised the kept pages a second time. It now serialises each page once and joins the single-page arrays. pages_json writes each page independently of its neighbours, so the bytes are the same, and this is checked below, including a truncated reply.
parser::parse_stream cloned each token, including its String, before matching. Space, comment and plain-word tokens are now handled while borrowed, ahead of the clone. Each branch is a copy of the matching arm, guard included.

Not touched: the incremental xref / document_global_state design (another lane is writing that proposal), render-pipeline and its vendored compiler, and the FT-070 PRs.

Why

Issue #65 is about IDE latency on large documents. After #535, protocol::handle_line still cost ~20 ms more than a bare compile on the 302-page article: output clone, double serialisation, and page clones. Clean layout still copied every word's text once more than needed on every convergence pass.

Timings

Apple M5 Pro, 18 cores, release build. The machine was still shared. vm.loadavg 1-minute load was 23.2–24.7 during the run (it was 90–106 for #535's numbers). That is quieter but not quiet, and wall-clock times remain noisy. Process CPU time is included.

302-page document (637 761 bytes, 5 912 blocks). Base and new ran alternately for 5 rounds. Each cell is the median over those rounds of the per-round median of 5 (large_doc_bench 5):

path	#535 head ms	this PR ms	speedup
first compile in process	83.4	78.1	1.07x
cold, `compile_full_project`	50.1	43.3	1.16x
cold, `handle_line`	69.9	49.1	1.42x
one-char edit, warm `Session` (full recompile)	62.0	52.0	1.19x
one-char edit, warm `handle_line`	70.6	51.3	1.38x
process user CPU for `large_doc_bench 5`, s	1.72	1.33	1.29x

The warm Session row uses the public owned-result API, which still makes one copy of the output. Only handle_line uses the borrowed path.

edit_latency_bench 20, one run each, load ~21–30. p50 in ms, shown as #535 head → this PR:

scenario	session p50	protocol p50
HW1 type in paragraph	0.309 → 0.271	0.624 → 0.328
HW1 type in inline math	0.407 → 0.266	0.570 → 0.336
HW1 type in display eq	0.299 → 0.261	0.483 → 0.327
HW1 delete/restore line	0.366 → 0.275	0.464 → 0.330
500KB type in paragraph	22.8 → 19.7	37.8 → 23.6
500KB type in inline math	23.1 → 19.3	39.8 → 23.7
500KB type in display eq	23.6 → 21.2	35.8 → 24.9
500KB delete/restore line	22.1 → 22.0	35.5 → 24.3

Remaining profile after this PR, self time including allocator work attributed to the nearest compiler frame, is diffuse:

emit ~8% (the one text.clone() a TextItem needs);
place ~6%;
IncrementalExpander::edit ~5%;
math layout ~8%;
shaped_width ~6%.

Output identity

fixture_digest (from #535) was built from #535 head 3c2b0e11 and from this branch and run over fixtures/, crates/compiler/tests/ and the generated 302-page document: 274 lines in total. For each file it prints SHA-256 digests of three things: the {:#?} dump of compile_full_project, the handle_line reply bytes, and the warm Session output after a one-character edit.

diff digest-base.txt digest-new.txt   -> empty, IDENTICAL (273 fixtures + 302-page doc)
302-page doc: f6e5442e201c7fdf e74f3665f2e8fe58 fb5e8304fa897317 pages=302 diags=0 (both)

Extra checks:

Truncated reply (exercises the rewritten bound_pages): a 2 000-section generated document (2.8 MB, 1 336 pages). Its reply is cut at the 8 MiB frame with the "were not delivered" diagnostic. fixture_digest lines are identical: 965b17a3b69df944 a99c86d672c4e5f8 bad9b098878e751d.
large_doc_bench reply sha256 (cold and warm handle_line), identical in all 10 runs, base and new: 51e4d9a9dc3f0ac0aa8cc90267fb8a73664650d2ebf246bcbb2958ad1cba9747.
edit_latency_bench reply digests, all 8 scenarios identical base vs new: ff68a4a2…, 625f5986…, 668dd9e7…, 12aeec9d…, ec8b2ffd…, 293ec170…, 826769f0…, e2119f31…. These cover warm protocol reuse on 500KB through the borrowed path.

Overlap check

gh pr list --search "perf": compiler: 3x faster large-document compile — shaping memo, glyph-table index, lent cleveref config (#65) #535 (this PR's base). The FT-070 PRs FT-070: engine performance harness, committed baseline and a >5% regression gate #206, FT-070 memory: measure the breakdown, derive the per-glyph carets (Cluster 160 -> 88 B) #232, render-pipeline, mac: display-list-v2-compact — derivable carets/hit_rects, run-level sources #257, FT-070: profile the warm keystroke, and stop copying a restored block's item list #273, FT-070 memory: pages reference the assembled items instead of cloning them (-244 MiB at 2 MB) #283, FT-070: a negotiated page window — the fix for a 500 KB document that has no reply #294 and FT-070 memory: a source range names its document by index (Cluster 88 -> 56 B, -105 MiB at 2 MB) #299 touch render-pipeline, perf harness and CLI, and none is edited here.
39 open PRs touch crates/compiler/src/{parser,layout,incremental,protocol}.rs. I grepped their diffs for the edited hunks (placed.extend, fn render_prepared_block, contains_key(text), output: output.clone(), previous.output.clone, let input = self.t[self.i].clone, incremental.output.pages, fn bound_pages): no matches.
coord.py claims: GH-COMPILER-PERF-2 is held by daniel-parent. GH-INCREMENTAL-XREF-DESIGN (another lane) is design-only, and this PR does not change reuse or xref behaviour.

Test results

cargo test (crates/compiler, debug): 60 suites, passed=671 failed=0 ignored=6
cargo test --release --test large_document_timing -- --ignored: test result: ok. 1 passed; 0 failed
large document: 302 pages; cold 102.1 ms; one-character edit 75.8 ms   (loaded machine, first-run)
fixture_digest: 273 fixture files + 302-page doc IDENTICAL to #535 head 3c2b0e11
fixture_digest: 2000-section (truncated reply) doc IDENTICAL
edit_latency_bench: 8/8 reply digests identical
rustfmt --check: no new drift in edited hunks (files already drift on origin/main with this toolchain)

Not done

Remaining SipHash cost of the shaping memo (~3.5%). The only clean fix is a faster hasher, and none is in this crate's dependency tree. Adding rustc-hash, or hand-rolling one, is left for a reviewer to decide.
The owned Session::compile_project_with result still deep-copies the output. There are no users outside the crate: bridge, paragraph-layout and flashtex-cli do not use Session, and render-pipeline uses its vendored copy. Callers can move to compile_project_borrowed when they are touched.
Parsing remains ~25% of a cold compile. Most of it is spread across command, finish_math and the expansion cache (IncrementalExpander::edit, convert_range); only the token clone was fixed here.
These timings are not from a quiet machine (load ~24 on 18 cores).
The vendored render-pipeline compiler copy is not synced.

🤖 Generated with Claude Code

https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

What changed: - src/bin/large_doc_bench.rs: deterministic ~300-page article (sections, labels/refs, cites + thebibliography, equation/align*, itemize/enumerate, tabular); median-of-N cold and one-character-edit timings through Session and protocol::handle_line, plus a reply digest. - src/bin/fixture_digest.rs: per-fixture SHA-256 of the clean compile dump (blocks, diagnostics, pages), the protocol reply, and a warm edited Session output, for byte-identity checks across perf changes. What was run: large document: 637761 bytes, 450 sections, 5912 blocks, 302 pages, 0 diagnostics, profile=release fixture_digest over fixtures/ and crates/compiler/tests: 273 files, two runs identical Next step: profile and fix the top hotspots. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…config (#65) What changed: - layout: shaping a Core 14 face is a pure function of (face, text); the width/source-bounds/missing-glyph summary layout reads is memoised per thread (reset past 131072 entries). Every cross-reference pass used to reshape every word with a fully allocated `Shaped`. - char_table: sorted binary-search index equal to `iter().find` (first occurrence) for lm_math/newcm_math/amssymb advances and the export Symbol/WinAnsi tables; math_font walked ~440 entries per ASCII character. - layout: inline_box (every tabular entry) lends the cleveref config to the detached cursor instead of building the default name table and cloning. What was run: fixture_digest over fixtures/ + crates/compiler/tests (273 files) and the 302-page doc: identical to origin/main 36fe7ec cargo test (debug): 59 suites, passed=671 failed=0 ignored=5 cargo fmt --check: ok; clippy -D warnings: 20 pre-existing errors, none in touched code Next step: interleaved A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

#65) What changed: - The previous commit accidentally included a local `cargo fmt` of the whole crate (43 unrelated files). Every file outside the perf change is restored to origin/main 36fe7ec; the perf edits are re-applied without reformatting. No behaviour change relative to the previous commit. What was run: git diff 36fe7ec --stat: 10 files changed, 607 insertions(+), 42 deletions(-) fixture_digest (273 files + 302-page doc): identical to origin/main Next step: interleaved A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…pile (#65) What changed: - tests/large_document_timing.rs: #[ignore] timing test sharing the bench's 302-page generator; asserts >= 300 pages, no diagnostics, and warm edit == clean compile. - large_doc_bench prints the first compile in the process (empty memo). What was run: cargo test --release --test large_document_timing -- --ignored: 1 passed (302 pages) cargo test (debug): 60 suites, passed=671 failed=0 ignored=6 Next step: A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…e pages once (#65) What changed: - layout: clean layout and every cross-reference pass call a new non-collecting render_block; only incremental reuse collects placed items. - layout: the shaping memo hashes a word once per hit (get, not contains_key plus index). - incremental: Session::compile_project_borrowed lends the retained output; compile_project_with clones from it as before. protocol uses the borrow. - protocol: bound_pages serialises each page once and joins the result instead of serialising, cloning and serialising again. - parser: parse_stream handles space, comment and word tokens borrowed before cloning the token. What was run: fixture_digest (273 fixtures + 302-page doc) vs #535 head 3c2b0e1: IDENTICAL large_doc_bench reply sha256 51e4d9a9dc3f0ac0aa8cc90267fb8a73664650d2ebf246bcbb2958ad1cba9747 (same as base) large_doc_bench 5 (load ~44): cold session 50.9 -> 40.6 ms, cold protocol 68.7 -> 46.7, edit protocol 69.9 -> 50.0 Next step: full test run, interleaved median-of-5 timings, PR stacked on #535. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

d-q222 · 2026-09-15T08:59:50Z

daniel-parent independent review (subagent)

Verdict: APPROVE

Method: detached worktrees for #535 (3c2b0e11) and #579 (657c883a) heads, CARGO_BUILD_JOBS=4, CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-review-58y.

Diff scope matches the PR body exactly. git diff review-535..review-579 --stat -- crates/ touches only crates/compiler/src/{layout,incremental,parser,protocol}.rs; fixture_digest.rs is byte-identical between the two heads.
Byte-identical output, verified myself, not just trusted. Built fixture_digest from both heads (saved each binary immediately after building to avoid a shared-CARGO_TARGET_DIR binary clobber — the two builds otherwise silently overwrite the same target/debug/fixture_digest). Ran both against every fixture under fixtures/ and tests/visual-corpus/fixtures/ (77 files: clean-compile dump, handle_line reply, and warm-edit dump each). diff of the two 77-line digest listings: identical.
Session::compile_project_borrowed is sound. Its elided output lifetime ties to &mut self under Rust's standard method-elision rule (multiple input lifetimes + a &mut self receiver → output borrows from self); this is enforced by the borrow checker, so it cannot compile if the borrow escaped. compile_project/compile_project_with's signatures are byte-for-byte unchanged (same source lines in both heads); their behavior is equivalent to before — previously the code cloned once for storage and returned the original, now it moves the original into storage and clones once on the way out via compile_project_with — same one deep copy either way. Grepped every other caller of compile_project/compile_project_with (render-pipeline's vendored copy, all the bench binaries, fixture_digest, tests): none call compile_project_borrowed directly, only protocol::compile does.
bound_pages/pages_json refactor (serialize each page once) is correct. pages_json always emits [...] with the brackets at offset 0 and len-1 with no extra whitespace, so bound_pages's one[1..one.len()-1] bracket-stripping-and-rejoining is safe; this is exercised by the handle_line/protocol digest above, which stayed identical.
Timing spot-check (large_doc_bench 3 300, run twice each for stability): reply sha256 identical between compiler: 3x faster large-document compile — shaping memo, glyph-table index, lent cleveref config (#65) #535 and compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535) #579 in every run. compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535) #579 was consistently faster: cold protocol 47.6→31.3 ms (≈1.52x), edit protocol 47.5→33.2 ms (≈1.43x) — in the same range as the PR's own 1.42x/1.38x on its larger benchmark, given this is a noisy shared machine.

No issues found.

GoKubar · 2026-09-15T22:37:45Z

Rolled into integration PR #658 (branch agent/kabir-claude/integration-q3) with #579, #640, #440, #621 and #358, merged with git merge --no-ff so commits and authorship are preserved. Conflicts were resolved once and the result verified once: 203 test binaries, 0 failures (compiler 902, render-pipeline 464, pdf 120, math-layout 74, tex-expansion 92), --no-run compile check first and separately, --locked builds clean, generated inventory regenerated rather than hand-merged. Merge #658 rather than this PR; #658's description records how each conflict in this branch was resolved.

Third instance of the same combination bug, found by re-running the full suite after the rebase rather than assuming it was mechanical. #593 (landed on main via #649) makes a source newline end the line under \obeylines: the lexer folds a lone newline into TokenKind::Space, and parse_stream_body's Space arm turns it into an Inline::LineBreak. #579's borrowed fast path, written before #593 existed, matched Space first and skipped it, so crates/compiler/tests/obeylines.rs went 7 passing -> 2 passing / 5 failing. Take the fast path for a Space only when \obeylines is not in force; a Comment is still always skipped. Under \obeylines spaces fall through to the slow path, which is where the newline is read from the token's own source bytes. cargo test --release --test obeylines: 7 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu

…e consumer What changed: main took #358's producer in via the integration commit 83285e3 ("integration: compiler + pdf + v2 queue (#579, #640, #440, #621, #358)"), so every Rust/protocol/docs/python file this branch carried is now already on main byte-for-byte. `git diff MERGE_HEAD -- crates/ protocol/ docs/ tests/ tools/` is down to the one producer-side piece main does NOT have: crates/document-runtime/src/lib.rs's capability allowlist. Conflicts and how they were resolved: - crates/render-pipeline/src/delta.rs, src/display.rs: took main. The branch's test helpers predate FT-070 (#294), which added `window` and `document_features` to DisplayList and turned `Page` into `Page::resident`. main's copies of `diag_list` / `header_only_diag_list` are the same tests with the new struct shape, and they pin the same Appendix A vectors (c4e7c712… on, e554935e… off). - apps/mac/Sources/FlashTeXMac/DisplayListDelta.swift: both sides changed `installed(from:pageBytes:lineBytes:)`. Kept main's windowed-frame guard and its doc comment, and re-applied this branch's `diagnosticsCapability:` flag, which is what gates `suggestion` into the header digest. Kept from this branch (nothing main provides): - crates/document-runtime allowlist: `display-list-v2-diagnostics` accepted only alongside `display-list-v2`, with the pairing/unknown-cap tests. - the whole Mac consumer: RenderingV2/RenderingV2Fast decode the optional `suggestion` (and tolerate labels/notes/help, which the proposal defines and main's producer documents as not yet on the wire), ShellModel.producerDiagnostics maps the live v2 frame through `asRuntimeV1` when compile_result.diagnostics is empty, and the delta digest hashes `suggestion` only when the capability was negotiated. Also dropped a stray blank line the branch had added to EditorDiagnostics.swift. What was run: - CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-merge363 CARGO_BUILD_JOBS=4 cargo test --manifest-path crates/document-runtime/Cargo.toml lib: test result: ok. 18 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out chunks: test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out display_sibling: test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out session: test result: ok. 19 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out - apps/mac: swift build -> Build complete! (19.60 sec) - apps/mac: swift test --filter 'RenderingV2|PreviewV2|EditorDiagnostics|ProblemsPanel|DisplayListDelta|V2Image' Executed 98 tests, with 12 tests skipped and 0 failures (0 unexpected) in 8.688 seconds Next step: push fast-forward to the PR branch and let CI run the full mac suite. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…lash-tex#440, flash-tex#621, flash-tex#358) (flash-tex#658) * wip(flash-tex#277): reproduce from_compiler dropping code and suggestion Add display::Diagnostic.suggestion (None at every constructor) and tests that expect unknown_command/\alpha and unsupported_feature. from_compiler still hardcodes code "compiler" and does not copy suggestion. Ran: CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd cargo test --manifest-path crates/render-pipeline/Cargo.toml --lib from_compiler_forwards Result: test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 89 filtered out display::tests::from_compiler_forwards_code_and_suggestion: left "compiler" right "unknown_command" Next: forward compiler code (explicit, else default_code, else "compiler") and suggestion; emit suggestion only in runtime-v1 JSON; extend corpus owner_for; keep v2 diagnostics without a suggestion key. Open questions: none in-scope. vendor/, apps, protocol schema untouched. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#277): forward compiler diagnostic code and suggestion from_compiler copies the compiler's code (explicit, else default_code(message), else "compiler") and suggestion. runtime-v1 JSON emits suggestion only when set; display-list-v2 diagnostics are unchanged except the code value. owner_for accepts the compiler code set; tikz_pipeline rejects those codes too so a forwarded unknown_command cannot slip through as "not compiler". Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - --lib from_compiler_forwards -- writer_matches_value_tree: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 88 filtered out - --lib write_json_matches_the_value_tree: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 89 filtered out - --test compiler_diagnostic_forward: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - --test tikz_pipeline: test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - python3 -m unittest tools.real-world-corpus.test_run: Ran 6 tests in 0.008s OK Next: full cargo test --manifest-path crates/render-pipeline/Cargo.toml control-vs-after. Open questions: no in-scope golden embeds "code":"compiler" for a compiler diagnostic (vendor/pdf fixture and apps/mac display-list fixture are out of lane). docs/user/compiler.md still shows the old example. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#277): propose display-list-v2-diagnostics capability Add protocol/proposals/display-list-v2-diagnostics.md: capability accepted only with display-list-v2; suggestion/labels/notes/help use the runtime-v1 shapes with v2 #/$defs/source; omit-when-empty; deltas carry the same keys; without the cap bytes stay identical. Schema delta is text in the proposal (frozen rendering-v2.schema.json untouched). labels/notes/help wait for a vendor re-pin past flash-tex#346; this lane serialises suggestion only. Ran: none (docs-only). Next: failing tests for negotiated vs not, compact vs JSON-tree, and a suggestion-only delta. Open questions: none in-scope. No vendor/, schema, apps, or contracts edits. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#277): reproduce v2 suggestion omitted without diagnostics cap CAP_DIAGNOSTICS and Wire.diagnostics / Capabilities.diagnostics, negotiated only with display-list-v2. Writers still emit the frozen four-key diagnostic, so capability-on output is byte-identical to capability-off and a suggestion change does not change the delta header digest. Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): cargo test --manifest-path crates/render-pipeline/Cargo.toml --lib diagnostics_capability_gates_suggestion -- suggestion_change_is_hashed -- negotiation_accepts_only_known Result: test result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured; 89 filtered out - negotiation_accepts_only_known_requested_capabilities: ok - diagnostics_capability_gates_suggestion_on_the_wire: left == right (no suggestion key) - suggestion_change_is_hashed_and_deltaed_only_when_serialised: header_digest equal with cap on Next: emit suggestion from write_diagnostics / diagnostic_json when the cap is on and the value is Some; hash it only then. Open questions: none. labels/notes/help still wait for flash-tex#346 re-pin. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#277): emit v2 suggestion when display-list-v2-diagnostics is on write_diagnostics and diagnostic_json_wire add suggestion after sources when Wire.diagnostics is set and the value is Some (BTreeMap order; never null). Deltas use the same writer. header_digest hashes suggestion only then, so a suggestion-only change produces a delta. Frozen writers stay four-key. No labels/notes/help slots (vendor compiler has none until flash-tex#346). Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - --lib diagnostics_capability_gates_suggestion -- suggestion_change_is_hashed -- write_json_matches: test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 89 filtered out - --lib writer_matches_value_tree -- from_compiler_forwards -- negotiation_accepts: test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 89 filtered out - --test compiler_diagnostic_forward: test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out Next: full cargo test --manifest-path crates/render-pipeline/Cargo.toml. Open questions: none in-scope. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * render-pipeline: from_compiler forwards exactly the compiler's code, no re-derived fallback Adopts the semantics of GoKubar's duplicate flash-tex#370: the compiler's constructors already set `code` via `default_code`, and a `None` is deliberate (request validation), so re-deriving `default_code(message)` here could label a diagnostic the compiler's own runtime-v1 reply leaves uncoded. Now `d.code.map_or("compiler", ...)`. --lib from_compiler 1/0 (test updated: uncoded stays "compiler"); compiler_diagnostic_forward 2/0 (tikz still unsupported_feature, set by the compiler's constructor); tikz_pipeline 3/0; golden_v1 1/0. Implementation-Agent: Claude Code (daniel-parent, mac-m5pro-dq222) Commit-Executor: Claude Code (daniel-parent) Lane-Owner: daniel-parent (mac-m5pro-dq222) Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: accept kernel preamble declarations as no-ops, warn on DocumentMetadata keys \NeedsTeXFormat, \ProvidesClass, \ProvidesPackage and \ProvidesFile are .cls/.sty declarations with no visible output in a document compiler, so they're accepted silently (required group, then the optional [info]/[date] bracket, both discarded). \DocumentMetadata (LaTeX2e 2022+) is different: its keys (pdfstandard, pdfversion, lang, testphase, ...) really do change real LaTeX's PDF output, which this compiler doesn't generate that way. Before \documentclass it's accepted with a warning naming the ignored keys; after \documentclass it's a real error, matching real LaTeX. Reuses the existing document_class: Option<String> field (set when \documentclass itself is processed) for the position check -- no new preamble-tracking mechanism needed. Regenerates the inventory artifacts and the Mac bundled copy. Implementation-Agent: muse-spark-1.3-contributor (Muse Code, lane kernel-text-symbols, slice 1) Commit-Executor: daniel-muse-lead (Claude Sonnet) Reviewed-by: daniel-muse-lead (Claude Sonnet) Muse-Lane-Head: 3dbf61de Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> * wip(flash-tex#358): negotiated-case protocol test next to flash-tex#354's omit-suggestion compiler_diagnostic_forward.rs: keep typo_alpah_is_unknown_command_with_alpha_suggestion_in_v1_only (v2 omits suggestion when the cap is off). Add negotiated_v2_diagnostics_emits_suggestion through handle_line: capability off omits suggestion; with display-list-v2 + display-list-v2-diagnostics the sibling carries suggestion and echoes the cap; the cap alone is rejected. Proposal status no longer says stacked on flash-tex#354. Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - cargo test --manifest-path crates/render-pipeline/Cargo.toml --lib test result: ok. 112 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - cargo test --manifest-path crates/render-pipeline/Cargo.toml --test compiler_diagnostic_forward --test display_list_delta compiler_diagnostic_forward: test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out display_list_delta: test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - python unittest tests/test_rendering_v2.py: Ran 23 tests in 0.037s OK Next: push this lane branch. Open questions: none. document-runtime allowlist remains flash-tex#363. Implementation-Agent: cursor-agent cursor-grok-4.6 Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#358): failing tests for overflow dependents, estimate, empty suggestion Reproduction tests for the independent review: declining display-list-v2 must also drop display-list-v2-diagnostics (FLASHTEX_MAX_REPLY_BYTES=6000); the size estimate with diagnostics off must match a suggestion-stripped list; Some("") must be omitted from the writer and the digest. Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - cargo test --lib estimate_omits_suggestion test result: FAILED. 0 passed; 1 failed (784 vs 758) - cargo test --lib empty_suggestion test result: FAILED. 0 passed; 2 failed - cargo test --lib empty_suggestion_is_not_hashed test result: FAILED. 0 passed; 1 failed - cargo test --test compiler_diagnostic_forward declining_display_list test result: FAILED. 0 passed; 1 failed (echoed ["display-list-v2-diagnostics"] with no sibling) Next: drop dependents with one helper, gate the estimate on Wire, omit empty suggestions in writer and digest, document dl2-canon-1 conditional suggestion, make the LM skip visible. Open questions: digest scheme — document conditional suggestion under dl2-canon-1 (images already Wire-gate the same scheme) rather than rename to dl2-canon-1+diagnostics (Mac still only knows dl2-canon-1). Implementation-Agent: cursor-agent cursor-grok-4.6-high-fast Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * compiler: brace-aware DocumentMetadata keys, seen-documentclass flag Fixes all three of daniel-parent's re-review points on flash-tex#440: 1. \DocumentMetadata{testphase={phase-III,math,table}} now warns naming only "testphase" (a top-level key with a braced value), not three separate "keys" -- keys are now split on top-level commas, tracking brace depth character by character, instead of a plain comma split over the raw (brace-stripped) argument text. 2. A new seen_documentclass flag (set on any \documentclass invocation, even an empty one) replaces document_class.is_some() for the \DocumentMetadata position check, so \documentclass{}\DocumentMetadata{lang=en-US} is now the real "must come before \documentclass" error instead of a warning. 3. missing_required_argument_is_a_parse_error now asserts the exact message this PR's own dispatch arm produces, instead of a generic "command not supported" message the unmodified parser already emits for any unknown preamble command. Mutation-tested for all three (revert -> new/strengthened test fails -> restore -> passes), independently spot-checked by the reviewer for fix 1 (disabling brace-depth tracking fails document_metadata_nested_braces_are_a_single_key as predicted). cargo test --test kernel_preamble_declarations: 8 passed, 0 failed. Full cargo test in crates/compiler: 0 FAILED across all suites. Implementation-Agent: muse-spark-1.3-contributor (Muse Code, lane fix440, slice 1) Commit-Executor: daniel-muse-lead (Claude Sonnet) Reviewed-by: daniel-muse-lead (Claude Sonnet) Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> * wip(flash-tex#358): drop dependents on overflow; omit empty suggestion; document digest Declining display-list-v2 now drops the whole family (images, device-color, diagnostics, delta, only) via is_display_list_family. The size estimate charges suggestion only when Wire.diagnostics would serialise it. Some("") is omitted from the compact writer, JSON tree, and header_digest through Diagnostic::wire_suggestion. Digest scheme stays dl2-canon-1: optional negotiated fields are Wire-gated like image items, not a rename (Mac still only accepts dl2-canon-1). LM skip in compiler_diagnostic_forward is now an assert. Stale runtime-v1 "does not forward code/suggestion" wording updated. Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - cargo test --manifest-path crates/render-pipeline/Cargo.toml --lib test result: ok. 115 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - cargo test --test compiler_diagnostic_forward --test display_list_delta compiler_diagnostic_forward: test result: ok. 4 passed; 0 failed; 0 ignored display_list_delta: test result: ok. 3 passed; 0 failed; 0 ignored - python unittest tests/test_rendering_v2.py: Ran 23 tests in 0.039s OK Next: git fetch origin && git merge origin/main, re-run, push. Open questions: none on the five review items. Implementation-Agent: cursor-agent cursor-grok-4.6-high-fast Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * wip(flash-tex#358): pin Appendix A suggestion digest; drop overflow dependents Appendix A header_digest now hashes suggestion at the same position and with the same encoding as delta.rs (only when display-list-v2-diagnostics is negotiated and the value is non-empty). Pinned c4e7c7129994d1b73c8dfe3d9b1b9a0cbf0edc49e7f9e6bc848d8c66e0126bb6 from a hand walk of that algorithm; diagnostics-off in-memory suggestion matches the no-suggestion digest. Overflow no longer echoes -images or -device-color. Ran (CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-diagfwd): - cargo test --manifest-path crates/render-pipeline/Cargo.toml --lib --test display_list_delta --test compiler_diagnostic_forward lib: test result: ok. 128 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out compiler_diagnostic_forward: test result: ok. 5 passed; 0 failed; 0 ignored display_list_delta: test result: ok. 3 passed; 0 failed; 0 ignored - /tmp/flashtex-schema-venv/bin/python -m unittest tests.test_rendering_v2 Ran 24 tests in 0.039s OK Next: push this checkpoint for daniel-parent re-review of the last PARTIAL. Open questions: none. Implementation-Agent: cursor-agent cursor-grok-4.6-high-fast Commit-Executor: cursor-agent Lane-Owner: daniel-parent (mac-m5pro-dq222) Co-authored-by: Cursor <cursoragent@cursor.com> * fix440: keep text glued after ']', honour escaped commas in DocumentMetadata - optional_bracket_argument consumes exactly up to the matching ']' at brace depth 0 and rewrites any trailing tail back into the token stream (fixes ProvidesFile{foo.cfg}[2024]VISIBLE losing VISIBLE). The helper predates this PR (on main since 01fb13c); all ~25 call sites benefit. - document_metadata treats a standalone one-character Word ',', '{', '}' (i.e. control symbols) as escaped literals via placeholders while splitting top-level commas (fixes foo=hello\,world reporting 'world'). - document_metadata_after_documentclass_is_an_error now asserts this PR's exact ordering message, not just any error mentioning DocumentMetadata. Implementation-Agent: muse-spark-1.3-contributor Commit-Executor: muse-spark-1.3-contributor * compiler: fix two real DocumentMetadata/bracket-argument bugs, merge main Fixes daniel-parent's third re-review of flash-tex#440: 1. `optional_bracket_argument` (parser.rs): [ and ] are not lexer-special characters, so `[2024]VISIBLE` lexes as one Word token, and the old code consumed the whole token, dropping `VISIBLE`. Now scans token-by-token for the closing `]` at brace depth 0, and rewrites any unconsumed tail back into the token stream in place. Confirmed pre-existing beyond DocumentMetadata: ~25 call sites share this helper (cite, bibitem, linebreak, footnote, newtheorem, lstlisting, ...), all fixed at the helper. Also newly correct: a `]` inside a nested `{...}` group no longer closes the bracket argument early. 2. `document_metadata` (parser.rs): a control symbol like `\,`/`\{`/`\}` lexes as its own standalone one-character Word token, while a real separator character is always embedded inside a longer Word token -- a reliable signal this fix uses to hide escaped characters behind private-use-area placeholders during the top-level comma/brace scan, restoring them afterward, so \DocumentMetadata{foo=hello\,world, lang=en} no longer reports "world" as a spurious key. 3. document_metadata_after_documentclass_is_an_error now asserts the exact ordering-error message, not any error mentioning DocumentMetadata. Merged current main and regenerated all inventory artifacts via the project scripts (not hand-merged). Verified independently, including a mutation check on fix 2 (disabling the escaped-comma placeholder fails document_metadata_escaped_comma_is_ not_a_separator as predicted; restoring passes): - cargo test --test kernel_preamble_declarations: 10 passed, 0 failed. - cargo test --test supported_latex: 8 passed, 0 failed. - Full cargo test in crates/compiler: 0 FAILED across all suites. Implementation-Agent: muse-spark-1.3-contributor (Muse Code, lane fix440v2, slice 1) Commit-Executor: daniel-muse-lead (Claude Sonnet) Reviewed-by: daniel-muse-lead (Claude Sonnet) Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> * compiler: split DocumentMetadata's plain top-level commas regardless of whitespace A standalone Word(",") from the lexer was always treated as an escaped `\,`, so `\DocumentMetadata{foo=bar , lang=en}` lost `lang`: a real separator comma surrounded by spaces lexes identically to an escaped one. The lexer gives both the same TokenKind::Word variant, so distinguish them the way optional_bracket_argument already does one function away: an escaped control symbol's source span is longer than its one-character text (it includes the backslash), while a plain word's span length always equals its text length. Applied the same check to `,`, `{`, `}`. Added regression tests for the shared optional_bracket_argument bracket reader across citation, footnote, \\[<length>], theorem-head and listing option callers, proving trailing text still survives after the bracket closes. Implementation-Agent: muse-spark-1.3-contributor Commit-Executor: daniel-muse-lead (Claude Sonnet) Reviewed-by: daniel-muse-lead (Claude Sonnet) Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> * compiler: document macro-comma limit, keep only real ]-glued regression tests Documented the accepted macro-produced-comma limitation (a comma from expanding a user macro carries the macro invocation's span, so the span-length check can't tell it apart from a real separator) at the span-length check site. Replaced the caller-regression tests added for optional_bracket_argument with ones that actually exercise the bug this PR fixes: text glued directly after the closing ']', not after a following required argument. The previous \cite/\footnote tests put TEXT after a required {...}, so they passed even on the pre-fix helper and proved nothing; verified against the pre-fix helper directly and kept only the ]-glued cases that fail on it (\ProvidesFile{f}[i]TEXT), since \\[<length>] and \item[<label>] already had their own tail-rewrite predating this PR. Implementation-Agent: muse-spark-1.3-contributor Commit-Executor: daniel-muse-lead (Claude Sonnet) Reviewed-by: daniel-muse-lead (Claude Sonnet) Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> * compiler tests: pin text glued after \\[2pt] line-break length skip_line_break_length has its own tail-rewrite for [2pt]TEXT-style glued text, separate from optional_bracket_argument; this pins that behavior next to the sibling ]-glued coverage. Implementation-Agent: muse-spark-1.3-contributor * compiler tests: pin text glued after explicit item label item_label_argument has its own tail-rewrite for [x]TEXT-style glued text, separate from optional_bracket_argument; this pins that behavior next to the other explicit-label coverage in list_structure.rs. Implementation-Agent: muse-spark-1.3-contributor * compiler: large-document bench and fixture output-identity digests (flash-tex#65) What changed: - src/bin/large_doc_bench.rs: deterministic ~300-page article (sections, labels/refs, cites + thebibliography, equation/align*, itemize/enumerate, tabular); median-of-N cold and one-character-edit timings through Session and protocol::handle_line, plus a reply digest. - src/bin/fixture_digest.rs: per-fixture SHA-256 of the clean compile dump (blocks, diagnostics, pages), the protocol reply, and a warm edited Session output, for byte-identity checks across perf changes. What was run: large document: 637761 bytes, 450 sections, 5912 blocks, 302 pages, 0 diagnostics, profile=release fixture_digest over fixtures/ and crates/compiler/tests: 273 files, two runs identical Next step: profile and fix the top hotspots. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: memoise Core 14 shaping, index glyph tables, lend cleveref config (flash-tex#65) What changed: - layout: shaping a Core 14 face is a pure function of (face, text); the width/source-bounds/missing-glyph summary layout reads is memoised per thread (reset past 131072 entries). Every cross-reference pass used to reshape every word with a fully allocated `Shaped`. - char_table: sorted binary-search index equal to `iter().find` (first occurrence) for lm_math/newcm_math/amssymb advances and the export Symbol/WinAnsi tables; math_font walked ~440 entries per ASCII character. - layout: inline_box (every tabular entry) lends the cleveref config to the detached cursor instead of building the default name table and cloning. What was run: fixture_digest over fixtures/ + crates/compiler/tests (273 files) and the 302-page doc: identical to origin/main 36fe7ec cargo test (debug): 59 suites, passed=671 failed=0 ignored=5 cargo fmt --check: ok; clippy -D warnings: 20 pre-existing errors, none in touched code Next step: interleaved A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: undo whole-crate reformatting swept into the previous commit (flash-tex#65) What changed: - The previous commit accidentally included a local `cargo fmt` of the whole crate (43 unrelated files). Every file outside the perf change is restored to origin/main 36fe7ec; the perf edits are re-applied without reformatting. No behaviour change relative to the previous commit. What was run: git diff 36fe7ec --stat: 10 files changed, 607 insertions(+), 42 deletions(-) fixture_digest (273 files + 302-page doc): identical to origin/main Next step: interleaved A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * pdf: exact route rounds colours to xcolor's 5 decimals and paints TikZ path items What changed: - v2: `pdf_number` formats an f64 as a bounded PDF number (shortest decimal rounded half away from zero, no exponent, -0 -> 0); sRGB colour components use 5 digits (pdflatex measured: RGB{20,80,170} -> 0.07843 0.31374 0.66667 rg). Replaces the exact-binary requirement that refused 0.8 and 0.07843. - v2: `path_fill` / `path_stroke` items (with clips) become q [rg RG] [clip W n] [w M d J j] path f|f*|S Q, in pgf's pdfTeX order. - exact: `M` (miter limit) joins the bounded operator set. What was run: - cargo test (crates/pdf): lib 53 passed; tests/v2 7 passed; all targets 0 failed - cargo check --tests (crates/rendering-core): ok Next step: re-pin crates/render-pipeline/vendor/pdf so `flashtex build` uses it. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * pdf: exact route paints alpha through pgf's /pgf@CA and /pgf@ca ExtGStates What changed: - exact: Op::StrokeAlpha / Op::FillAlpha, written `/pgf@CA<a> gs` and `/pgf@ca<a> gs`, the names and one-key dictionaries pgf's pdfTeX driver writes (measured with pdflatex 1.40). `gs` joins the bounded operator set for those names only; any other ExtGState is still refused. Validation refuses an alpha outside [0, 1] and `gs` during path construction. Each page declares the states it selects once, sorted by name, inline: `/ExtGState << /pgf@ca0.4 << /ca 0.4 >> >>`. - v2: a paint alpha below 1 is no longer an error. It is rounded like a colour component and selected right after the colour inside the item's `q … Q`: path_stroke sets CA, path_fill, rule and glyph_run set ca. - Tests: exact operator stream and resources for stroke and fill alpha, per-page dedup, refusal of other gs names, text alpha on a glyph run. What was run: - cargo test (crates/pdf): test result: ok. 53 passed; 0 failed (lib), 12 passed (tests/exact), 5 (images), 11 (navigation), 28 (render), 3 (type1), 10 passed; 0 failed (tests/v2). - cargo check --tests (crates/rendering-core): Finished. Next step: build extended/tikz-clipping-patterns with a scratch copy whose render-pipeline/vendor/pdf is this crate. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: ignored large-document timing test; bench reports first compile (flash-tex#65) What changed: - tests/large_document_timing.rs: #[ignore] timing test sharing the bench's 302-page generator; asserts >= 300 pages, no diagnostics, and warm edit == clean compile. - large_doc_bench prints the first compile in the process (empty memo). What was run: cargo test --release --test large_document_timing -- --ignored: 1 passed (302 pages) cargo test (debug): 60 suites, passed=671 failed=0 ignored=6 Next step: A/B timings, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: stop copying placed text, lend the session output, serialise pages once (flash-tex#65) What changed: - layout: clean layout and every cross-reference pass call a new non-collecting render_block; only incremental reuse collects placed items. - layout: the shaping memo hashes a word once per hit (get, not contains_key plus index). - incremental: Session::compile_project_borrowed lends the retained output; compile_project_with clones from it as before. protocol uses the borrow. - protocol: bound_pages serialises each page once and joins the result instead of serialising, cloning and serialising again. - parser: parse_stream handles space, comment and word tokens borrowed before cloning the token. What was run: fixture_digest (273 fixtures + 302-page doc) vs flash-tex#535 head 3c2b0e1: IDENTICAL large_doc_bench reply sha256 51e4d9a9dc3f0ac0aa8cc90267fb8a73664650d2ebf246bcbb2958ad1cba9747 (same as base) large_doc_bench 5 (load ~44): cold session 50.9 -> 40.6 ms, cold protocol 68.7 -> 46.7, edit protocol 69.9 -> 50.0 Next step: full test run, interleaved median-of-5 timings, PR stacked on flash-tex#535. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * pdf: support TikZ tiling patterns What changed: - Add deterministic PatternType 1 tiling resources and typed /Pattern cs, scn operators to the exact PDF route. - Generate the eight requested PGF cells with pattern color, accept the v2 path_fill pattern extension, and cover PaintType 2 RGB selection. - Add operator, resource, parser, deterministic-output, structure-check, and Ghostscript-backed sample verification coverage. What was run: - CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-pattern CARGO_BUILD_JOBS=4 cargo check --manifest-path crates/pdf/Cargo.toml - CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-pattern CARGO_BUILD_JOBS=4 cargo test --manifest-path crates/pdf/Cargo.toml --lib --tests - CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-pattern CARGO_BUILD_JOBS=4 cargo test --manifest-path crates/vector-graphics/Cargo.toml --lib tikz::tests::unsupported_input_is_reported_not_dropped - /opt/homebrew/bin/gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=png16m -r144 on /private/tmp/flashtex-pattern-sample.pdf Next step: - Carry pattern name/color state through vector-graphics, render-pipeline display items, and display-list JSON so the live TikZ corpus can populate the new PDF contract. Implementation-Agent: codex gpt-5.6-luna (max) Commit-Executor: codex (this session) Lane-Owner: daniel-parent (mac-m5pro-dq222) * compiler: stop dropping six amsfonts symbols on a false premise gen_amssymb.py carried SKIP = {angle, hbar, mho, sqsubset, sqsupset, rightleftharpoons}, justified as "kernel commands amsfonts only redefines keep their kernel glyphs". That was wrong twice over (checked against pdfTeX 3.141592653-2.6-1.40.27, TeX Live 2025): - \mho, \sqsubset, \sqsupset are not kernel commands at all: latex.ltx makes them \not@base stubs and amsfonts.sty 99-101 is what provides them. FlashTeX diagnosed them "not supported in math mode" even with amssymb loaded. - \angle, \hbar, \rightleftharpoons are kernel composites that amsfonts replaces with one msam/msbm glyph of different metrics (\angle 6.37344pt -> 7.22223pt, \hbar 5.76172pt -> 5.40280pt at 10pt). With the package loaded, \angle/\hbar fell through to bare COMMAND_GLYPHS rows at Latin Modern advances matching neither pdflatex answer, and \rightleftharpoons was "not supported". Removing SKIP lets the generator emit all six with the class and slot amsfonts declares (provider Amsfonts, so \usepackage{amsfonts} alone suffices, matching the .sty). \hbar needs a MANUAL entry: amsfonts puts it on msbm "7E, the slot amssymb also gives \hslash, and unicode-math names that character only as \hslash. Since the original draft, main gained the package gate (543bde5) with kernel-width arms for unloaded \angle/\hbar; those keep working and the loaded case now takes the declared msam/msbm rows. \rightleftharpoons - whose comment said "no row at all, so nothing to gate yet" - gets the same treatment: a gated kernel arm (Rel, U+21CC, 1.000002em, the measured 10.00002pt \mathpalette stack) so an unloaded document does not regress to a missing-package error for a command base LaTeX2e does define. scripts/amssymb_api.rs.in is caught up with 84ff3c4's hand-edit of the generated file (CharTable index instead of linear find), so `python3 scripts/gen_amssymb.py > src/amssymb.rs` is byte-identical again; verified before this change the generator reproduced main's table except for that drift, and after it the diff is exactly the six rows plus their LM advances. Regenerated docs/user/compiler.md, supported/supported-latex.json, supported/coverage.md and the Mac bundled copy via scripts/render_supported_latex.sh. Tests: tests/amssymb_skip.rs reconstructs pdfTeX's \show mathchars ("340A "045C "057E "3440 "3441 "0566) from the generated class, font and slot, and pins the msam/msbm widths against the pdflatex \hbox measurements. angle_and_hbar_keep_the_kernel_composite_without_amsfonts extended to the loaded-case msam/msbm advances and to \rightleftharpoons. Salvaged from PR flash-tex#212 (engine/amssymb-generator-skip), rebuilt against current main (post-543bde5c package gating). Implementation-Agent: fable-salvage Commit-Executor: fable-salvage (direct; Cursor usage limit) * compiler: regenerate supported-latex docs on the merged tree docs/user/compiler.md (generated) conflicted with the flash-tex#641 integration batch; the merge took main's copy and this regenerates it via scripts/render_supported_latex.sh on the merged tree, restoring the six amssymb rows on top of main's new inventory (555 -> 559 math entries). supported-latex.json and the Mac bundled copy were already correct; sync-supported-latex.sh --check is byte-identical (704a3974fe88). gen_amssymb.py re-verified byte-identical against src/amssymb.rs after the merge. Gates re-run on the merged tree: cargo test --release --no-run clean; --lib 380 passed 0 failed; amssymb_skip 1 passed; supported_latex 8 passed; lm_math_binding 6 passed. Implementation-Agent: fable-salvage Commit-Executor: fable-salvage (direct; Cursor usage limit) * integration fix: flash-tex#579's borrowed fast path passes control_symbol_kern's post-flash-tex#576 arguments flash-tex#579 predates flash-tex#576 (kern control symbols measured by definition bytes); its new parse_stream_body fast path called the old 3-argument form. Pass maps_to_invocation and definition from the token entry, as the slow path does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu * integration: regenerate supported-latex artefacts (render_supported_latex.sh; sync-supported-latex.sh) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu * integration fix: flash-tex#579's fast path must not shadow main's tabbing control arm flash-tex#642 added a \\=/\\>/\\< tabbing arm to parse_stream_body's match that runs before the control_symbol_kern arm. flash-tex#579's borrowed fast path, written before flash-tex#642, matched those words first and typeset them as text, so tabbing produced 0 tab stops instead of 2 (5 failures in crates/compiler/tests/tabbing.rs). Mirror the slow path's guard. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu * integration fix: perf-bench names flash-tex#358's new diagnostics capability main's crates/perf-bench (flash-tex#206) constructs v1::Capabilities literally; flash-tex#358 adds the diagnostics field, so the bench stopped compiling (E0063) and the engine performance job failed at Build. Pass diagnostics: false, matching the bench's other opt-in capabilities. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu * integration fix: flash-tex#294's page window and flash-tex#358's diagnostics wire meet flash-tex#294 replaced Page's items field with PageContent + an accessor, added DisplayList::{window, document_features}, and added tests/page_window.rs; flash-tex#358 adds Wire::diagnostics. Neither sees the other on its own branch: - delta.rs/display.rs test helpers built Page and DisplayList literally (E0560, E0063 x2) -> Page::resident(..) and the two new fields as None - tests/page_window.rs's const WIRE misses diagnostics (E0063) - tests/math_symbols.rs took Page::items as a field (E0615) -- broken on main itself, fixed separately in flash-tex#664 and carried here cargo test --release --no-run on crates/render-pipeline: 0 errors. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu * integration fix: flash-tex#579's fast path must not swallow an \obeylines newline Third instance of the same combination bug, found by re-running the full suite after the rebase rather than assuming it was mechanical. flash-tex#593 (landed on main via flash-tex#649) makes a source newline end the line under \obeylines: the lexer folds a lone newline into TokenKind::Space, and parse_stream_body's Space arm turns it into an Inline::LineBreak. flash-tex#579's borrowed fast path, written before flash-tex#593 existed, matched Space first and skipped it, so crates/compiler/tests/obeylines.rs went 7 passing -> 2 passing / 5 failing. Take the fast path for a Space only when \obeylines is not in force; a Comment is still always skipped. Under \obeylines spaces fall through to the slow path, which is where the newline is read from the token's own source bytes. cargo test --release --test obeylines: 7 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Xd5Hmwh5GHNTiAmHUJ1MZu --------- Co-authored-by: d-q222 <279808976+d-q222@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

d-q222 added 5 commits September 14, 2026 23:18

d-q222 mentioned this pull request Sep 15, 2026

FT-003: dispatch to mac-claude-a — acknowledgement required #2

Closed

GoKubar mentioned this pull request Sep 15, 2026

integration: compiler + pdf + v2 queue (#579, #640, #440, #621, #358) #658

Merged

GoKubar closed this in #658 Sep 15, 2026

GoKubar deleted the agent/daniel-parent/compiler-perf-2 branch September 16, 2026 04:18

d-q222 mentioned this pull request Sep 16, 2026

mac: consume display-list-v2-diagnostics — v2 suggestions reach Problems and Tab-to-fix (after #358) #363

Open

Navigation Menu

compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535) - #579

compiler: handle_line 1.4x faster on large documents — no placed-item copies, borrowed session output, pages serialised once (stacked on #535)#579
d-q222 wants to merge 5 commits into
mainfrom
agent/daniel-parent/compiler-perf-2

d-q222 commented Sep 15, 2026

Uh oh!

d-q222 commented Sep 15, 2026

Uh oh!

GoKubar commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

d-q222 commented Sep 15, 2026

What

Why

Timings

Output identity

Overlap check

Test results

Not done

Uh oh!

d-q222 commented Sep 15, 2026

Uh oh!

GoKubar commented Sep 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants