render-pipeline: page and paragraph control from the compiler's penalty, discretionary and parameter nodes — needs #568 re-pin (#64) by d-q222 · Pull Request #569 · flash-tex/flashtex

d-q222 · 2026-09-15T07:35:20Z

What

Render-pipeline side of issue #64. It consumes the compiler nodes from #568, so pdflatex's line and page breaks come out for the page and paragraph control commands.

Paragraphs (adapter.rs, typeset.rs):
- Inline::Penalty becomes a penalty item. A forced one ends a justified line (\linebreak, \penalty-10000); \\ stays \hfil\break. \linebreak/\nolinebreak move the space before them after the penalty, and blanks after the command are eaten as \@ifnextchar / \ignorespaces do.
- Inline::Discretionary becomes a flagged break (\hyphenpenalty, or \exhyphenpenalty when the pre-break text is empty) with its pre-break text. It is invisible unbroken, and the word next to it gets no automatic points (TeX §899).
- Inline::PagePenalty becomes an undiscardable zero-width anchor plus a \vadjust penalty after the line it lands on (new VBlock::vadjust_penalty).
Vertical list: Block::Penalty becomes the next paragraph's penalty_before (lowest of a run). \filbreak gets \vfil/\vfilneg around it (new VBlock::fil_break). pagebreak is dropped from the source-scanned PAGE_BREAKS, so a \pagebreak inside a paragraph no longer splits it.
Parameters (adapter::BreakOverrides, read where each paragraph ends, with group scope from Parsed::parameters): \tolerance, \pretolerance, \emergencystretch, \hfuzz (so \sloppy, \fussy, sloppypar), \widowpenalty, \clubpenalty, \interlinepenalty (\samepage). Also \raggedbottom/\flushbottom (last top-level one) and \hyphenation exceptions added to the English patterns (Stylesheet::hyphenation).
\cleardoublepage in the body of a two-sided document starts an odd page, through the same open_right test \chapter uses (Doc::double_page_starts).
adapter.rs builds the lowered group LineBreak through line_break_inline, as the re-pinned compiler has skip_pt (compiler: report \\[<dimen>]'s optional argument on Inline::LineBreak #478).

This PR does not build yet, by design. crates/render-pipeline builds against the frozen vendor/compiler snapshot, which has none of #568's nodes. This is the same compiler-PR-plus-blocked-pipeline-PR pattern as #514/#515. Against the current pin: 15 E0599 (no Penalty/PagePenalty/Discretionary variants), 3 E0609 (no parameters/hyphenation fields), 1 each of E0425/E0432/E0433 (BreakParameter, ParameterAssignment).

Why

Before, per pdflatex (full audit table below): \- was painted, \pagebreak in a paragraph split it and added a page, \linebreak was ragged, \pagebreak[1] forced a page, and \nobreak/\allowbreak/\penalty/\goodbreak/\filbreak/\nopagebreak did nothing. Every breaking parameter except a top-level \sloppy was ignored, \hyphenation was ignored, and \cleardoublepage in book never added the empty page.

Overlap check

Same as #568: no open PR or claim implements these commands. Muse lanes #495/#504 (\addvspace) and the integration lane's #479/#480/#483 were not touched. #356 (topskip/\brokenpenalty) also edits pagebuild.rs/typeset.rs VBlock code; the new VBlock fields are additive. #515 (cancel) is the other re-pin-blocked pipeline PR.

Test results

Built and run in a scratch export of this branch with vendor/compiler replaced by #568's crates/compiler and --features linebreak-skip (the repository's vendor/ is untouched):

cargo test --release --test page_para_control: test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
same test file on origin/main's pipeline: test result: FAILED. 0 passed; 10 failed (written before the 11th, cleardoublepage, which also fails there: book gives 2 pages)
cargo test --release --no-fail-fast (whole crate): 384 passed; 5 failed
  the 5 are bundled_typewriter, which reads apps/ files the export leaves out: on origin/main in the real tree "test result: ok. 5 passed"
--test tie / hyphenated_fragment_units / par_leading_oracle / line_break_skip_macro: all ok
pdflatex probe harness (PyMuPDF line/page comparison): 86 of 89 match (origin/main's binary: 34 of 89)
cargo clippy --all-targets: 65 findings on the crate, none in added lines

tests/page_para_control.rs pins, to pdflatex outcomes: the tie-like \nobreak/\nolinebreak/\penalty10000 lines, the \allowbreak break, justified \linebreak/\penalty-10000 (477.5 bp vs \\ 207.8 bp), \- and \discretionary{=}{}{} breaking and invisible, \hyphenation exceptions, and the \sloppy/sloppypar/\tolerance/\emergencystretch/\fussy/group-scope lines. On pages: the vertical penalties, in-paragraph \pagebreak, \widowpenalty=0/\clubpenalty=0/\interlinepenalty/\samepage page-1 line counts, the \flushbottom/\raggedbottom baselines, and the \cleardoublepage page counts.

After the re-pin, re-run the whole suite in the real tree before merging.

Not done

The vendor/compiler re-pin itself (integrator action).
\enlargethispage / *: the page builder has no per-page goal change. It needs a goal item in VItem or a per-page side table in both page loops (break_pages_regions and the float/notes builder). pdflatex holds 47 lines on page 1, FlashTeX 46.
\looseness: paragraph-layout's LineBreakParams has no looseness (a paragraph-layout change plus its own re-pin).
Under \flushbottom, a page ended by a vertical \pagebreak (no \vfil) is not stretched: the compiler reports \newpage and \pagebreak[4] as the same Block::PageBreak.
\discretionary with non-empty post-break or no-break text sets the no-break text with no break: the line painter pairs one record with each pre-break run only.
Parameters assigned in an \input file are not ordered against paragraphs of another file. Parameters are not applied inside float bodies (their own \@parboxrestore values win).
\nobreakdash: untouched.

Audit: page and paragraph control commands (GH-64)

State on origin/main @ 36fe7ec3, measured with flashtex check/build (render pipeline, vendor compiler pin c95977d6) against pdflatex (TeX Live 2026, article 10pt, \pagestyle{empty}). Line and page outcomes were read from both PDFs with PyMuPDF (words grouped by baseline) on engineered documents; each engineered probe was checked to discriminate (pdflatex sets different lines/pages without the command). Node values come from pdflatex \showlists/\showthe.

"After" is the compiler PR plus the render-pipeline draft, built together in a scratch export with vendor/compiler replaced by the compiler branch (the repository's vendor/ is untouched).

Usage = files using the command among the 12,038 .tex files of TeX Live 2026 texmf-dist/doc (real documents; the in-repo corpus has almost none of these commands).

Command	Usage (files)	Before (origin/main)	After	pdflatex evidence
`\newpage`	1348	supported	unchanged	`a\newpage b`: 2 pages (FlashTeX 2)
`\clearpage`	1043	supported	unchanged	`a\clearpage b`: 2 pages (FlashTeX 2)
`\-`	452	wrong: the hyphen is painted even when the line does not break (`manu-scripts`)	`Inline::Discretionary`; a flagged `\hyphenpenalty` break with the face's hyphen, invisible unbroken; the word gets no automatic points (TeX §899)	unbroken: `manuscripts`; engineered: line 1 ends `qwrtzp-`, with 2 more `xx` in front `qwrtzplkjhgfd` starts line 2
`\pagebreak[n]`	431	wrong: inside a paragraph it splits the paragraph and starts a page; `[n<4]` still forces a page	vertical mode: `Block::PageBreak` (n=4) / `Block::Penalty` `-\@getpen{n}`; in a paragraph `Inline::PagePenalty` after the line it is set on	`a\pagebreak{} b` and `a\pagebreak[1] b`: one line `a b`, 1 page (FlashTeX 2); `a\par\pagebreak[1] b`: 1 page (FlashTeX 2); `\showlists`: `\vadjust` `\penalty -10000`
`\nobreak`	297	unsupported (`unknown_command` error)	`Inline::Penalty`/`Block::Penalty` 10000	`Figure\nobreak\ 7`: line 1 ends `Figure 7`; between paragraphs 46/47: page 1 ends `Line 45.`
`\penalty`	217	unsupported; the number is typeset as text (`Fig-ure10000`)	node in either mode, span covers the number	`\penalty10000\` : `Figure 7`; `\penalty-10000` after `Line 43.`: page 1 ends there
`\sloppy` / `\fussy` / `sloppypar`	212 / 21 / 41	partial: a top-level `\sloppy` only (source scan); `\fussy` unknown; `sloppypar` warned and ignored	`Parsed::parameters` (tolerance 9999, emergencystretch 3em, hfuzz .5pt / 200, 0, .1pt) with the group end that restores them; read where each paragraph ends	`sloppypar`: `floccinaucini-` / `hilipilification ... hippopotomon-`; `\sloppy\fussy`: `floccinaucinihilip-`
`\hyphenation{}`	200	unsupported diagnostic, exceptions ignored	`Parsed::hyphenation`; the pipeline adds them to the English patterns	`\hyphenation{flocc-inaucinihilipilification}`: line 1 ends with the whole word (control: `floccinaucinihilip-`)
`\enlargethispage` / `*`	187	unsupported (plus an error for `\baselineskip`)	compiler: `EnlargeThisPage {pt, shrink}` (pt, cm, ..., `n\baselineskip`), no diagnostic. Pipeline: not applied	`\enlargethispage{2\baselineskip}`: page 1 holds 47 lines; FlashTeX 46
`\linebreak[n]`	173	wrong: a forced break is set ragged like `\\`; `[n<4]` ignored	`Inline::Penalty` `-\@getpen{n}` with the space before it moved after it; a forced break ends a justified line	line 1 `Short words here` ends at x 477.5 bp (FlashTeX before: 207.8, the `\\` position); `\showlists`: `\penalty -151` for `[2]`
`\cleardoublepage`	157	wrong when two-sided: no empty page	compiler unchanged (`Block::PageBreak`); pipeline: the odd-page test `\chapter` already uses	`book`: `a\cleardoublepage b` 3 pages (FlashTeX 2); `article`: 2
`\tolerance` / `\pretolerance`	134 / 53	unsupported, value typeset as text	parameters, read per paragraph	`\tolerance=9999`: `floccinaucini-` (control `floccinaucinihilip-`); `\pretolerance` probe did not discriminate
`\raggedbottom` / `\flushbottom`	130 / 32	unknown commands; only the class default	`FlushBottom` parameter; the last top-level one decides the page glue	42 lines + an unbreakable paragraph: last baseline y 627 (article) → 675 with `\flushbottom`; `book` 674 → 626 with `\raggedbottom`
`\nopagebreak[n]`	115	accepted no-op (the page could still break there)	`\@getpen{n}` penalty, both modes	between `Line 46.` and `Line 47.`: page 1 ends `Line 45.` (FlashTeX before `Line 46.`)
`\widowpenalty` / `\clubpenalty`	97 / 79	unsupported, value typeset as text	parameters, used by the page builder	4-line paragraph after `Line 43.`: 45 lines on page 1, 46 with `\widowpenalty=0`; after `Line 45.`: 45, 46 with `\clubpenalty=0`
`\emergencystretch`	91	unsupported; `\setlength{\emergencystretch}` warned	parameter (both forms)	`\emergencystretch=3em`: `floccinaucini-`
`\allowbreak`	84	unsupported	`\penalty0` node	`Figure/\allowbreak seven/`: line 1 ends `Figure/`
`\discretionary{}{}{}`	63	unsupported; the three arguments typeset as text	`Inline::Discretionary` (plain text of each argument); pipeline breaks with the pre-break text. Non-empty post-break/no-break text: the no-break text is set with no break (painter limitation)	`qwrtzp\discretionary{=}{}{}lkjhgfd`: line 1 ends `qwrtzp=`
`\interlinepenalty`	58	unsupported	parameter	`\interlinepenalty=10000`: the 4-line paragraph moves whole, page 1 holds 43 lines
`\looseness`	54	unsupported, value typeset as text	compiler: `Looseness` parameter, no diagnostic. Pipeline: not applied (paragraph-layout has no looseness)	probe did not discriminate
`\goodbreak` / `\filbreak`	46 / 31	unsupported	`\par` + `Block::Penalty` -500 / -200 with `\vfil`…`\vfilneg`	after `Line 43.`: page 1 ends `Line 43.` (FlashTeX before `Line 46.`)
`\samepage`	32	unsupported (command and environment)	`InterlinePenalty(10000)` for the group	the paragraph after `Line 43.`/`44.` moves whole to page 2
`\nolinebreak[n]`	22	accepted no-op (the break stayed legal)	`Inline::Penalty` `\@getpen{n}`, space moved after it	`Figure \nolinebreak 7`: `Figure 7` (FlashTeX before: `Figure` / `7 shows`)
`\nobreakdash`	6	unsupported (amsmath)	unchanged	not done

Counts (24 rows). Before: supported 2, accepted no-op 2, partial 1, wrong 4, unsupported 15. After: 21 rows match pdflatex on their probes (one, \discretionary, with the post-break/no-break limitation); not done 3 (\enlargethispage and \looseness parsed by the compiler but not applied by the pipeline; \nobreakdash untouched). Still open: under \flushbottom a page ended by a vertical \pagebreak keeps natural glue (pdflatex stretches it; FlashTeX treats it like \newpage).

🤖 Generated with Claude Code

https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…ty, discretionary and parameter nodes (#64; needs the compiler re-pin) What changed: - Paragraphs: Inline::Penalty -> a penalty item (a forced one ends a justified line; \linebreak/\nolinebreak move the space before them after the penalty); Inline::Discretionary -> a flagged break with its pre-break text (\- and \discretionary{pre}{}{}), no automatic hyphenation of the word (TeX 899); Inline::PagePenalty -> a \vadjust penalty after the line it lands on. - Vertical list: Block::Penalty before a paragraph (\goodbreak, \filbreak with \vfil/\vfilneg, \pagebreak[n<4], \nopagebreak, \nobreak, \penalty); "pagebreak" is no longer a source-scanned paragraph split. - Parsed::parameters read where each paragraph ends: \tolerance, \pretolerance, \emergencystretch, \hfuzz (\sloppy, \fussy, sloppypar), \widowpenalty, \clubpenalty, \interlinepenalty (\samepage); \raggedbottom/ \flushbottom; Parsed::hyphenation added to the English patterns. - Body \cleardoublepage in a two-sided document starts an odd page. - adapter.rs: the group LineBreak uses line_break_inline (the skip_pt field). What was run (scratch export, vendor/compiler replaced by the compiler branch, --features linebreak-skip; repository vendor/ untouched): - --test page_para_control: test result: ok. 11 passed; 0 failed - same tests on origin/main's pipeline: test result: FAILED. 0 passed; 10 failed (before the cleardoublepage test was added) - full cargo test --release: 384 passed; 5 failed (bundled_typewriter only: its files are under apps/, which the export leaves out; 5 passed on origin/main) - 86 of 89 pdflatex probes match (enlargethispage x2, flushbottom after \pagebreak differ) Next step: re-pin vendor/compiler past the compiler PR, then merge this. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…s page What changed: The adapter lays an \include'd file's structural commands (\chapter, ...) out before that file's next unit (paragraph, heading, rule, picture). A file whose whole body is a captioned float has no unit, so its \chapter (heading, page clear, number) was never laid out and the floats piled onto one page ahead of later text. Likewise a trailing entry \chapter with only a float after it was dropped: the end-of-document loop handled only marks, page styles and \maketitle. - An entry \input/\include consumed without a unit of the file it reads (found from the reading order) lays that file's commands out in place, then \include's closing \clearpage. - A unit of an included file flushes up to the command that reads its file, not merely the next \input/\include. - Included files read so far are finished whenever the next file is read, not only when the entry file resumes. - The unit loop takes one last pass after the last unit, so commands after it (\chapter, \include, marks, \maketitle) go through the same code; the separate trailing loop is gone. tests/include_float_only_chapters.rs pins pages, lines and baselines against pdflatex for two and three float-only \include'd chapters, the same under \includeonly (with and without trailing text), and single-file float-only chapters. What was run: cargo test --release --test include_float_only_chapters test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out same test without the adapter change: test result: FAILED. 0 passed; 3 failed chapter_in_included_file: test result: ok. 2 passed; 0 failed float_body_oracle: test result: ok. 1 passed; 0 failed float_graphics_oracle: test result: ok. 1 passed; 0 failed float_notes_oracle: test result: ok. 1 passed; 0 failed float_numbering_oracle: test result: ok. 1 passed; 0 failed floats_oracle: test result: ok. 3 passed; 0 failed include_float_lists: test result: ok. 1 passed; 0 failed input_order_oracle: test result: ok. 1 passed; 0 failed toc_oracle: test result: ok. 1 passed; 0 failed cargo test --release -p flashtex-render-pipeline --no-fail-fast: passed 386, failed 0, ignored 3 Next step: a float right after a body \clearpage (no \include) still lands on the old page; that break is carried by the next block's eject and is left to the #569/#583/#584 page-break stack. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

…he readiness merges What changed: - New docs/integration/repin-runbook.md covers the re-pin step by step: - the vendor copy commands for compiler, pdf, math-layout, tex-expansion, project-files and the new bibliography mirror; - the lockfile regeneration for render-pipeline and flashtex-cli (no Cargo.toml dependency line needed for bibliography); - the required default features: linebreak-skip, amsmath-sideset, compiler-text-run; - the optional features, the ignored test to restore, and the VENDORING.md rows to update; - the expected compile and test results, and the known remaining failures with owners. - docs/integration/daniel-parent-preview.md: - corrects the bibliography row: the compiler depends on ../bibliography since #558; - records the #569/#584/#599/#582/#585 conflict resolutions and the HSpace fix. What was run (scratch re-pin, vendor from c47e145, CARGO_BUILD_JOBS=4, never committed): - render-pipeline cargo test --release --no-fail-fast: all binaries ok, 466 passed, 0 failed, 3 ignored - page_para_control: test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.04s - flashtex-cli cargo test --release: ok. 31 passed; 0 failed / ok. 26 passed; 0 failed; 1 ignored - amsmath TOTAL 59/59 within 0.5 bp; amssymb TOTAL 38/39 within 0.5 bp; tabular TOTAL 128/128 (words 0.5 bp, rules 0.1 bp); display-placement TOTAL 47/48 within 0.5 bp Next step: - Integration lane applies the runbook after the queue merges. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz

@caption

… (#646) * render-pipeline: tabular lengths keep their group scope; \small survives a float caption; @{} glue (#56) What changed: - Table lengths (\tabcolsep, \arrayrulewidth, \doublerulesep, \extrarowheight, and longtable's) are read as seen at the table: only assignments whose group ({...}, \begingroup, an environment) is still open count, in every form (\setlength{\x}{v}, \setlength\x{v}, \addtolength, \x=v). The old scan took the last brace-form \setlength anywhere in the file. - A float-level size declaration (\small before \caption) is carried into later content runs, as \@caption's \normalsize is inside its own group. - @{...} material keeps glue at both ends (@{\hspace{1em}}, @{\quad}). - \hspace{<n>em} is set as <n> quads of the current font, not the compiler's fixed-size em. - 17 new tabular corpus fixtures (107-123) with pdflatex references. What was run: - oracle.py check: main 117/123, this branch 123/123 (words 0.5 bp, rules 0.1 bp) - cargo test --release (render-pipeline): 79 test binaries ok, 0 failed - tabular_oracle: test result: ok. 1 passed; 0 failed - lib: test result: ok. 140 passed; 0 failed; 2 ignored Next step: compiler PR accepting table length assignments without the "not implemented here" warning / unknown \tabcolsep error. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: report/book number figures and tables within the chapter What changed: - floats::number follows report.cls/book.cls: \thefigure and \thetable are \ifnum\c@chapter>\z@\thechapter.\fi\@Arabic\c@figure, reset by \@addtoreset{figure}{chapter}. \chapter* and book's \chapter outside \mainmatter step nothing; \appendix zeroes the chapter counter and makes \thechapter \@alph. Captions and \ref both use the value. - FloatSpec.number is the printed string ("1.2"), not a bare counter. - New pdflatex-pinned fixtures in fixtures/float-numbering (report chapters with \chapter*, report appendix, book front/main/back matter, an article control) and tests/float_numbering_oracle.rs: caption label, page and position within 1 bp, and the resolved \ref line. What was run: - cargo test --test float_numbering_oracle: test result: ok. 1 passed; 0 failed - same test on origin/main sources: 17 mismatches (e.g. "Figure 1:" for pdflatex's "Figure 1.1:", chapter two's first figure printed "Figure 3") - cargo test --test floats_oracle: ok. 2 passed; float_body_oracle: ok. 1 passed; float_notes_oracle: ok. 1 passed; toc_oracle: ok. 1 passed; footnotes_oracle: ok. 1 passed; --lib floats: ok. 4 passed Next step: open the GH-69 PR with the object-stream PDF sizing commit. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: size PDF graphics whose page tree is in object streams What changed: - graphics::probe_pdf falls back to flashtex_pdf::reader when the byte scan fails on a PDF with /ObjStm (PDF 1.5 object streams, pdfTeX's default \pdfobjcompresslevel 2). Before, every such PDF was image_unavailable and took no space. CropBox/MediaBox/Rotate handling is shared (page_info). - New fixtures/float-graphics: a 72 dpi PNG and two pdfTeX object-stream PDFs (one /Rotate 90), made by make_fixtures.py; oracle.py pins box size and position from pdflatex. tests/float_graphics_oracle.rs checks width=\linewidth, scale, keepaspectratio, angle before/after a size key, \textwidth multiples, and the PDFs, within 1 bp, with no image diagnostic. - Unit test graphics::tests::pdf_object_streams_are_read. What was run: - cargo test --test float_graphics_oracle: test result: ok. 1 passed; 0 failed - same test without the graphics.rs change: 7 mismatches - cargo test --lib graphics: test result: ok. 8 passed; 0 failed Next step: [H] from the float package, then the PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: float.sty [H] sets the box exactly where it stands What changed: - FloatSpec::exact_here: with \usepackage{float}, a vertical-mode [H] float is not a float (\@float@HH). paginate splices \float@endH's \vskip\intextsep, the box and \vskip\intextsep at its marker, with TeX's interline glue (\lineskip against the line above; the next line's glue against the box's zero depth). It is never deferred or counted. - [H] in the middle of a paragraph or on a two-column figure* now has its own diagnostic and falls back to h. [H] without the package keeps its old one. - packages: float is implemented by the pipeline. \floatstyle, \restylefloat and \newfloat are still diagnosed by the compiler. - fixtures/floats: make_fixtures.py adds 11-float-h and 12-h-passes-deferred. oracle.py takes fixture names, so only those two were pinned; the other 10 sources and references are byte-identical. floats_oracle expects 12 fixtures, plus a diagnostics test for [H]. What was run: - cargo test --test floats_oracle: test result: ok. 3 passed; 0 failed (11-float-h 2/2 pages, 13 anchors, max 0.002 bp; 12-h-passes-deferred 3/3 pages, 9 anchors, max 0.003 bp) - same fixtures on the pre-change sources: 26 mismatches - cargo test --lib: test result: ok. 138 passed; 0 failed; 2 ignored - float_body_oracle, float_notes_oracle, footnotes_oracle, toc_oracle, multicol_oracle, twocolumn_title_oracle: ok, 1 passed each; graphics_draft_demo: ok. 3 passed; float_numbering_oracle: ok. 1 passed; float_graphics_oracle: ok. 1 passed Next step: open the GH-69 PR and post the verification table on #69. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: \chapter in an \include'd file is a chapter What changed: The adapter read structural body commands (\chapter, \part, \appendix, book matter, marks/page styles, \addcontentsline) from the entry document's source only. A \chapter in chapters/one.tex was set as a plain paragraph on the current page: no page clear, no number, no marks, no contents line. Included documents now get the same scan; their commands are laid out before that document's next unit, and any left after its last unit are laid out when the entry document resumes. \maketitle, \noindent, contents lists and nested \input stay entry-only. Corpus extended/project-book-include (#42): 5 pages -> 7 (pdflatex 7). What was run: cargo test --release (render-pipeline): passed 377 failed 0 ignored 3 test result: ok. 2 passed; 0 failed (chapter_in_included_file) test result: ok. 1 passed; 0 failed (input_order_oracle) test result: ok. 1 passed; 0 failed (toc_oracle) test result: ok. 20 passed; 0 failed (flashtex-cli cli) Next step: \include's own \clearpage before/after (article: pdflatex 3 pages, we set 1) and the missing CONTENTS running head in book front matter are separate follow-ups. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * pdf: an ellipsis's periods no longer make every period extract as "…" What changed: - v2 export reduced cluster text to ToUnicode by first use. The pipeline sets `\textellipsis` as three periods in one cluster whose text is "…", so the period glyph was mapped to U+2026 for the whole font: the ellipsis extracted as "………" and every later sentence-final period as "…" (measured on "em dash… ellipsis." in OT1 and T1+lmodern documents). - Only a one-glyph cluster now names its glyph's text. A glyph seen only inside clusters of several glyphs takes the character the font's own cmap maps to it (new `TrueTypeFont::char_for_glyph`, lowest code point), and the cluster text only when the cmap has none. Ligatures (`fi`, one glyph) still extract as their cluster text. - tests/v2.rs: `ellipsis_periods_keep_the_period_in_to_unicode` (fails on main with `{28: "a", 88: "…"}`). What was run: - cargo test --release --test v2 test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out - cargo test --release (crates/pdf, whole crate): 118 passed, 0 failed - flashtex build --v2 + flashtex-pdf-exact from-v2 on the survey documents d4-punct-ot1 / d5-punct-t1-lm: "em dash……… ellipsis…" -> "em dash... ellipsis." Next step: - Re-pin crates/render-pipeline/vendor/pdf so `flashtex build` and Export PDF pick it up (vendor re-pins are the integration lane's). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * math-layout: TeX's math font kerns (make_ord, tex.web §752) from the TFM lig/kern programs What changed: - tools/gen_cm_tfm.py emits each font's lig/kern program (lig_kern_starts, lig_kern, kerns) read from the real TFMs; cm_tfm.rs and ams_tfm.rs regenerated (additions only; existing char rows unchanged). - tfm.rs: TfmFont::lig_kern walks the program as TeX does (first matching instruction; a ligature shadows a later kern). - MathFontMetrics::ord_pair (default None) + CmMathMetrics impl: same-family check via symbol_slot, the scaled kern, and whether the font is a text font. - layout.rs: make_ord appends the kern after an unscripted Ord character followed by an Ord..Punct character of the same family, and drops the italic correction in a text font (§755). Bins demoted by Rule 6 are not kerned. What was run: - cargo test --test math_font_kerns: test result: ok. 2 passed; 0 failed - cargo test (math-layout): all suites ok (12, 4, 5, 28, 4, 2, 11 passed; 0 failed) Next step: feature-gated forwarding in render-pipeline's TexMathMetrics, then HW1 and amsmath corpus measurement. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: an em is the quad of the font in force; table lengths follow macro invocations (#525 review) What changed: - `\hspace{<n>em}`, `\quad` and `\qquad` in running text are now set in quads of the font current where they are read. `Item::Quad` carries a style: the series/family come from the source declarations at the command, the size from the compiler's sizes of the text around it (`glue_size`). Before, the quad was the paragraph's base font, so `{\Large Ff\hspace{2em}Gg}` was 8.165 bp short and `{\bfseries ..}` 2.989 bp short. - The blank after `\hspace{..}`'s closing brace is an ordinary space. It was being skipped as if it followed a control word (3.3-3.9 bp). - Size environments (`\begin{small}..\end{small}`, all ten sizes) give text, rules, kerns and tables their size unless a declaration made inside is still in force. A paragraph whose `\par` is read inside one gets that size's `\baselineskip`. The compiler's "environment is not implemented" warning for them is superseded. - `length_at` applies the `\setlength`/`\addtolength`/`\<len>=` a user macro makes at the top level of its body (nested macros too, `#k` from braced arguments) at the invocation, with the invocation's group scope. An assignment that cannot be read (optional argument, unbraced argument) is left out and reported as a `table_limitation` warning. - Corpus fixtures 124-126 (`\hspace{em}`/`\quad` under `\small`/`\normalsize`/`\Large`, `\bfseries` and size environments, in body text and tables, at 10/11/12pt), 127 (size-environment paragraphs) and 128 (macro `\setlength`/`\addtolength` before tables), with pdflatex references. Unit tests: table_lengths_follow_macro_invocations, size_environments_give_their_size_until_a_declaration, glue_takes_the_size_of_the_text_it_is_read_with. What was run: - python3 oracle.py check --render .../flashtex-render: TOTAL 128/128 (words 0.5 bp, rules 0.1 bp) - cargo test --release (crates/render-pipeline): 79 test binaries, all "test result: ok", 0 failed - lib: test result: ok. 143 passed; 0 failed; 2 ignored - tests/tabular_oracle.rs: tabular_oracle: 128/128 fixtures match pdflatex; test result: ok. 1 passed; 0 failed Next step: re-review of #525. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: math-font-kerns feature forwards math-layout's ord_pair (off until re-pin) What changed: - Cargo.toml: new off-by-default `math-font-kerns` feature; render-pipeline depends on vendor/math-layout (not crates/math-layout), which predates MathFontMetrics::ord_pair. - mathtex.rs: TexMathMetrics::ord_pair asks the embedded CM programs for families 1-3 and the installed rm-lmr TFM for family 0. - mathtext.rs: TextRunMetrics forwards, never kerning a \text handle. - tests/math_font_kerns.rs (feature-gated): glyph origins of $r,a,b$, $f,P,Y.$, $f(x)$ $\Gamma,\Delta$ $x_1,x_2$ $V_{a}$ and HW1's display against pdflatex within 0.02 bp. What was run (scratch copy of crates/render-pipeline with vendor/math-layout replaced by this branch's crates/math-layout, feature on; vendor/ untouched): - cargo test --release --features math-font-kerns --test math_font_kerns: test result: ok. 1 passed; 0 failed - same test with ord_pair forced to None: fails, $r,a,b$ glyph 1 +0.608 bp, $f,P,Y.$ glyph 5 +3.633 bp - amsmath corpus oracle.py check: TOTAL 59/59 within 0.5 bp (before and after) - rank.py --only hw1: page 2 within 0.5 bp 203 -> 212; the 9 kern words (lines 560.9/585.4) from 0.60-0.61 bp to <=0.005 bp; no other word moved by more than 0.005 bp - cargo build --release --bin flashtex-render against vendor/ (feature off): ok Next step: re-pin vendor/math-layout (integration lane), then make math-font-kerns a default feature. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * math-layout: keep the ord_pair imports and impl clear of #470's lines What changed: separate use lines for MathChar/OrdPair and CmMathMetrics::ord_pair moved after accent_sizes, so this branch merges cleanly with #470 (tag-textrun). No behaviour change. What was run: - cargo test (math-layout): every suite test result: ok, 0 failed Next step: open the PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: number floats in the order the \include tree is read What changed: - adapter::reading_order walks \input/\include from the entry file (compiler lookup: path, then path.tex; the preamble's last \includeonly; cycles and unknown files read nothing) into ordered byte ranges. - floats::number steps chapter/appendix/matter and the figure/table counters over those ranges, so an entry-file float after an \include whose file has its own \chapter gets 2.1, not 1.2. A float of a file never read (\includeonly-excluded) has no number, label, spec or .lof line. - LoF/LoT entries take the float's own number (report pre-chapter float is "1", not "0.1") and merge in reading order. - Float markers in paginate are placed by reading order, so an entry float after an included chapter follows that chapter's blocks. - \include issues its \clearpage before the file and after it. - Fixtures 05 (report, two files), 06 (book, \include + \input + appendix include), 07 (report, \includeonly) with pdflatex references; oracle.py handles multi-file projects; tests/include_float_lists.rs pins LoF/LoT/\ref against pdflatex's .lof/.lot/.aux. What was run: cargo test --release --test float_numbering_oracle test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out cargo test --release --test include_float_lists test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out cargo test --release --test toc_oracle test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out cargo test --release --test floats_oracle test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out cargo test --release --lib reading_order test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 140 filtered out Next step: full crate run, then PR stacked on #537 and #538. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * math-layout: math ligatures in make_ord and kerns/ligatures inside text runs (tex.web §752-§753) What changed: - `OrdPair` carries the pair's ligature instruction (`OrdLigature { op, ch }`); `CmMathMetrics::ord_pair` reports cmr's `=:` ligatures (ff, fi, fl, ffi, ffl, and the quote/dash/Spanish marks) as Unicode text characters, and `text_glyph` boxes those characters from their OT1 slots. - `layout`: the first pass runs `make_ord` over the list before boxing. A ligature rewrites the list as §753 does: `=:` merges the pair (the ligature takes the right atom's scripts), `=:|`/`|=:` replace one side, `|=:|` inserts, and the pair is retried unless the op is a `>` form. So `$\mathrm{f}\mathrm{i}$` is one fi glyph and `$\mathrm{f}\mathrm{i}^2$` is fi^2. - `make_text` (`Nucleus::Text`, the multi-character `\mathrm{..}` and operator runs) applies the same program between its characters: the font kerns (`AV`, `Tr`, `Wa`) and ligatures (`ff`, `ffi`), last italic kept. - tests/math_font_kerns.rs: 19 new widths pinned to pdfTeX `\wd0`, plus a synthetic program exercising every ligature form on both paths. What was run: cargo test --manifest-path crates/math-layout/Cargo.toml --test math_font_kerns test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out cargo test --manifest-path crates/math-layout/Cargo.toml test result: ok. 12 / 4 / 5 / 28 / 4 / 3 / 2 / 11 passed; 0 failed before this change the new widths fail, e.g. $\mathrm{AV}$ 15.13890pt vs pdfTeX 14.02777pt Next step: forward ligatures through render-pipeline's TexMathMetrics and one-character \mathrm groups; measure the pipeline against pdflatex. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: math ligatures and one-character alphabet kerns behind math-font-kerns What changed: - `TexMathMetrics::ord_pair` reads family 0's instruction straight from the installed rm-lmr TFM program (`Tfm::pair_program`, no text-mode boundary handling) and reports its ligatures, so `make_ord` forms them; `text_glyph` boxes a formed ligature (U+FB01 ...) from its OT1 slot. - A one-character math alphabet pair of the same alphabet (`\mathbf{T}\mathbf{o}`, `\mathit{A}\mathit{V}`) is kerned from that alphabet's text-font TFM. Ligatures there are not formed (no character addresses a ligature slot of those fonts). - `typeset`: a one-letter/digit `\mathrm{X}` is math-layout's `TextChar` (TeX §1186), so it joins its neighbour. All behind `math-font-kerns`. - math-layout `cm::ligature_char` shared by both providers. - tests/math_ligatures.rs pins glyph origins against pdflatex (lmodern). What was run (scratch copy, vendor/math-layout := this branch's crate): cargo test --release --features math-font-kerns --test math_ligatures test result: ok. 1 passed; 0 failed negative control on #560's overlay: fails, "$x\mathrm{A}\mathrm{V}x$: glyph 2 at x 140.215 bp, pdflatex 139.003 (+1.212)" cargo test --release --no-fail-fast --features math-font-kerns (whole crate): 377 passed; 0 failed; 3 ignored oracle.py check: TOTAL 59/59 within 0.5 bp (before and after, per-fixture lines identical) rank.py --only hw1: page 2 within 0.5 bp 212 of 215, pages 1 and 3 unchanged (same as #560) worktree, feature off: cargo check --release --tests ok Next step: open the PR stacked on #560. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * math-layout: keep the new layout.rs imports clear of #470's lines What changed: the OrdLigature/SizeClass/Cow imports move to a block of their own, so #560's import lines stay byte-identical and #470 still merges cleanly. What was run: git merge-tree --write-tree HEAD origin/agent/daniel-parent/tag-textrun: no conflict cargo test --manifest-path crates/math-layout/Cargo.toml test result: ok. 12 / 4 / 5 / 28 / 4 / 3 / 2 / 11 passed; 0 failed Next step: open the PR stacked on #560. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: a chapter whose body is only a float still clears its page What changed: The adapter lays an \include'd file's structural commands (\chapter, ...) out before that file's next unit (paragraph, heading, rule, picture). A file whose whole body is a captioned float has no unit, so its \chapter (heading, page clear, number) was never laid out and the floats piled onto one page ahead of later text. Likewise a trailing entry \chapter with only a float after it was dropped: the end-of-document loop handled only marks, page styles and \maketitle. - An entry \input/\include consumed without a unit of the file it reads (found from the reading order) lays that file's commands out in place, then \include's closing \clearpage. - A unit of an included file flushes up to the command that reads its file, not merely the next \input/\include. - Included files read so far are finished whenever the next file is read, not only when the entry file resumes. - The unit loop takes one last pass after the last unit, so commands after it (\chapter, \include, marks, \maketitle) go through the same code; the separate trailing loop is gone. tests/include_float_only_chapters.rs pins pages, lines and baselines against pdflatex for two and three float-only \include'd chapters, the same under \includeonly (with and without trailing text), and single-file float-only chapters. What was run: cargo test --release --test include_float_only_chapters test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out same test without the adapter change: test result: FAILED. 0 passed; 3 failed chapter_in_included_file: test result: ok. 2 passed; 0 failed float_body_oracle: test result: ok. 1 passed; 0 failed float_graphics_oracle: test result: ok. 1 passed; 0 failed float_notes_oracle: test result: ok. 1 passed; 0 failed float_numbering_oracle: test result: ok. 1 passed; 0 failed floats_oracle: test result: ok. 3 passed; 0 failed include_float_lists: test result: ok. 1 passed; 0 failed input_order_oracle: test result: ok. 1 passed; 0 failed toc_oracle: test result: ok. 1 passed; 0 failed cargo test --release -p flashtex-render-pipeline --no-fail-fast: passed 386, failed 0, ignored 3 Next step: a float right after a body \clearpage (no \include) still lands on the old page; that break is carried by the next block's eject and is left to the #569/#583/#584 page-break stack. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: \check@icl italic correction and nested list closing skips (display-placement 35/36) What changed: - An upright text font command (\textbf, \textup, \texttt, \emph in italic text, ...) and amsmath's \eqref (\textup{\tagform@{..}}) put the italic correction of the character before them under the interword space (latex.ltx \check@icl -> \maybe@ic -> \sw@slant). Fixture 17's `(2.1).` was 0.784 bp left (the `f` of "Ref", kern 0.7922 pt). - A list's closing \addvspace\@topsepadd uses the \topsep of the level that closes (was always level 1), with that list's own \partopsep flag; lists closing together and a following \item's \addvspace\itemsep keep the larger natural skip instead of the sum. Fixture 20's `2. Outer again` was 7.97 bp low. - A paragraph right after \end{itemize}/\end{enumerate} with no blank line is not indented (\@endpe), as after center/quote. - Pins: tests/text_command_left_italic_correction.rs, tests/nested_list_end_skips.rs (pdflatex numbers; all four fail on main by -0.791, +7.970, -1.993, +3.985 bp); display_placement.rs PASSING adds 17 and 20; baseline-render-pipeline.json refreshed. What was run: - display-placement oracle.py check: TOTAL 35/36 within 0.5 bp (main: 33/36; 15 still fails on the rich \tag, owned by #470/#585) - amsmath_corpus oracle.py check: TOTAL 59/59 within 0.5 bp - cargo test --release -p flashtex-render-pipeline (all targets): 81 test binaries, 379 passed, 0 failed, 3 ignored Next step: fixture 15 lands with #585 plus the math ToUnicode lane. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: math minus, asterisk and phi extract as the right Unicode What changed: - Math '-' extracts as U+2212 MINUS SIGN and '*'/\ast as U+2217 ASTERISK OPERATOR (both were ASCII); \phi/\varphi now extract as U+03D5/U+03C6 (they were swapped). Glyph selection and positions are unchanged. - Tests assert semantically correct Unicode, not pdflatex parity: pdflatex without the cmap package extracts cmex/cmsy glyphs as artefacts (\sum -> P, \mapsto -> 7->), which are deliberately not copied. What was run: - render-pipeline math targets: 68 passed, 0 failed - render-pipeline lib: 138 passed, 0 failed, 2 ignored - crates/pdf: all suites passed; git diff --check: passed Next step: - independent review. Implementation-Agent: codex gpt-5.6-luna (max) Commit-Executor: daniel-parent (committed Codex's working tree; sandbox blocked its git index) Lane-Owner: daniel-parent (mac-m5pro-dq222) Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: keep only the four semantic math extraction fixes What changed: - The previous commit accidentally shipped the first-pass table that copied pdflatex's cmex/cmsy extraction artefacts (\mapsto -> 7->, \Longrightarrow -> =>, \circ -> bullet, \setminus -> ASCII backslash, ...). This commit applies the corrected working tree: extraction_text only maps '-' -> U+2212, '*' -> U+2217 and swaps \phi/\varphi; every other glyph keeps main's text. Tests assert semantically correct Unicode. What was run: - render-pipeline math targets and lib (see next comment on #597) Next step: - re-run tests; update #597. Implementation-Agent: codex gpt-5.6-luna (max) Commit-Executor: daniel-parent (staging slip in the previous commit) Lane-Owner: daniel-parent (mac-m5pro-dq222) Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: long arrows extract as their drawn join pieces What changed: - The long arrows move out of common_math_glyph_runs_use_semantic_unicode into long_arrows_extract_as_their_drawn_pieces (option C from #536). - A two-glyph join (#536's drawing) must extract as `=⇒` / `−→`, matching pdfTeX's own PDF (`x =⇒y −→z`, pdflatex + PyMuPDF). While main still draws one Latin Modern Math glyph, the text is the single `⟹` / `⟶`. - A comment records that one `⟹` for two drawn glyphs needs PDF /ActualText marked content, a future crates/pdf change outside this PR. What was run: - math_symbols (debug): test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.46s Next step: - Merge into agent/daniel-parent/integration-preview and rerun math_symbols there. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * oracles: a math minus sign aligns with the references' hyphen-minus What changed: - tools/visual-oracle/rank.py `norm` folds U+2212 MINUS SIGN into U+002D before alignment. pdfTeX's own PDFs extract a math minus as U+2212 (PyMuPDF on pdflatex's 19-vmatrix.pdf: "= ad −bc"; 28-xrightarrow.pdf: "f−→B", "h←−"), which is what this branch now emits. The pinned references read "-" only because pdftext's glyph-name table maps `minus` to "-". The references and FlashTeX are both right; the word matcher was not. - amsmath_corpus/oracle.py `regroup` orders coincident glyphs (the \relbar and arrow head of \xrightarrow) with the same fold, so `−→` sorts like the references' `-→`. - test_rank.py: a minus sign aligns with a hyphen-minus. What was run: - amsmath oracle, this branch's flashtex-render: TOTAL 58/59 after the norm fold alone (28-xrightarrow order tie), TOTAL 59/59 within 0.5 bp with both - amsmath oracle, main's flashtex-render (36fe7ec): TOTAL 59/59 within 0.5 bp - amssymb oracle, main and this branch: TOTAL 38/39 within 0.5 bp (both; 32-braces-narrow fails identically on main) - tabular oracle, main and this branch: TOTAL 106/106 (words 0.5 bp, rules 0.1 bp) - display-placement oracle, main and this branch: TOTAL 33/36 within 0.5 bp (identical rows) - python3 -m unittest discover -s tools/visual-oracle -p 'test_*.py': Ran 14 tests, OK Next step: merge into the integration preview (#609) and rerun there. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * visual-oracle: candidate word text slices cluster byte ranges as bytes What changed: - rank.v2_words sliced the glyph run's Python str with the display list's cluster ranges, which are UTF-8 byte offsets. After the first non-ASCII character in a run (`∈`, and now this branch's U+2212 minus) every later glyph got its neighbour's text: `|f(x_k) − f(y_k)|` read as words `−f`, `(y` instead of `−`, `f(y`, and `x ∈ K` left an empty-text word. The sweep-2 "apparent regressions with identical pixels" (inline-math p1/p2, math-sheet p2, hw1 p1, conf-paper p2) were this. The amsmath oracle's own reader already encoded before slicing. - test_rank.py: the v2 fixture helper writes byte offsets; a run with `−` and `∈` keeps its word text. What was run (tools/visual-oracle/rank.py, all 22 real-world fixtures, same flashtex-pdf-exact; aligned words summed over every page): - old rank.py: main 36fe7ec 15285, preview 5d1eb06 15253 - this rank.py: main 15330, preview 15326 - pages whose aligned/within-0.5 counts differed main vs preview only through text (conf-paper p2, hw1 p1, inline-math p1/p2, math-sheet p1/p2, ps-calculus p2/p3, twelvept-plain p2, lecture-notes p2) are now identical - amsmath 59/59, amssymb 38/39, tabular 106/106, display-placement 33/36 (main) and 35/36 (preview): unchanged by this commit - python3 -m unittest discover -s tools/visual-oracle -p 'test_*.py': OK Next step: merge into the integration preview (#609). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * render-pipeline: read each table length once per adapt call, not per table (#525 review, #623) What changed: - `length_at_checked` scanned the source before every table (a CmdScan of the prefix, plus a `group_open_between` rescan per assignment) for each of the four tabular lengths and the five longtable lengths. On a document with many tables that is quadratic or worse: 800 tables spent 1.6 s in it. It is the same class of bug #623 fixed in `split_at_page_breaks`. - Within an adapt call (`MacroDefsScope`) each length is now indexed once per document: `length_assignments` collects every `\setlength`/ `\addtolength`/`\<name>=` and assigning macro invocation of the whole source once, `GroupTokens` lexes the group depth once (the tokens `group_open_between` counts, with the next-lower-depth token of each), and `LengthIndex` links each assignment to the last earlier one still in force at its end. A table binary-searches its byte offset and walks that chain past closed groups. The value and `unresolved` flag are folded in the same order as the scan, so the floats are bit-identical. - Outside an adapt call, at a byte inside a control word, or for a document whose assignments do not form a chain (an assigning macro in another's argument, an assignment ending inside a comment), the old scan runs unchanged as `length_at_scan`. - The indexes live in a new `lengths` field of `MacroDefsEntry`, placed before `index` so #623's `setlengths` field merges without conflict (`git merge-tree` of this commit with #623's head is clean). What was run: - `cargo test --release --lib -- table_length`: test result: ok. 5 passed; 0 failed - table_length_index_matches_the_prefix_scan: the indexed lookup equals the scan at every byte of 8 hand-written and 300 generated sources, for tabcolsep/arrayrulewidth/LTpre (both index and fallback paths hit). - table_lengths_scale_linearly_with_the_number_of_tables: before: 200 tables 41.2ms, 400 tables 242.7ms, 800 tables 1.637s (panicked: not linear) after: 200 tables 393µs, 400 tables 717µs, 800 tables 1.42ms - flashtex-render end to end, generated doc, best of 3 (before -> after, v2 identical): 200 tables 0.16s -> 0.08s; 400 tables 0.71s -> 0.22s; 800 tables 4.05s -> 0.73s - v2 JSON + compile_result of all 128 tabular_corpus fixtures and 22 fixtures/real-world documents, head 99defd5 vs this commit: `diff -rq` IDENTICAL (150/150) - `oracle.py check`: TOTAL 128/128 (words 0.5 bp, rules 0.1 bp) - `cargo test --release` (render-pipeline): exit 0, 79 test binaries, 383 passed; 0 failed; 3 ignored Next step: merge #623 after #525 (or the reverse); the rest of the remaining 800-table time is #623's `split_at_page_breaks` fix. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * integration: #639's layout_cache_keys test constructs #525's Item::Quad with its style field Main's tests/layout_cache_keys.rs (#639) predates #525's Item::Quad { em, style }; the test's shared TextStyle fills the new field. Implementation-Agent: fable-integration-rp Commit-Executor: fable-integration-rp (direct; Cursor usage limit) * integration: a file-tail float keeps its blank-line mask; include_float_lists expects #608's joined paragraph Two #578 x #608 interactions on the combined head: - floats::mask (#608) turned a float-only line into a % comment; for a float at the very tail of an \included file that left the paragraph open at EOF, and the compiler joined it with the entry document's next text, so #578's closing \clearpage landed after the merged paragraph (4 pages instead of pdflatex's 5). A float with nothing but blanks/comments after it now keeps the blank-line mask; entry documents are unaffected (\end{document} follows their floats). - tests/include_float_lists.rs (#578) expected "Refs ..." on its own line; under #608 the [h] floats stay inside the paragraph (pdflatex behaviour), so the refs share the paragraph's line. Implementation-Agent: fable-integration-rp Commit-Executor: fable-integration-rp (direct; Cursor usage limit) * integration: restore #543's entry-document chapter starts for \thelstlisting The #578 delta removed the chapter_starts push together with the chapter-based figure numbering it replaced, but #543's listing numbers (list_blocks' Lol entries and listings::apply captions) still come from chapter_numbers over those starts (toc_oracle fixture 37-report-lstlistoflistings). Implementation-Agent: fable-integration-rp Commit-Executor: fable-integration-rp (direct; Cursor usage limit) --------- Co-authored-by: d-q222 <279808976+d-q222@users.noreply.github.com>

d-q222 mentioned this pull request Sep 15, 2026

re-pin readiness: pipeline compiles against the queued compiler/pdf/math-layout (runbook for the integration lane) #631

Closed

Navigation Menu

render-pipeline: page and paragraph control from the compiler's penalty, discretionary and parameter nodes — needs #568 re-pin (#64) - #569

render-pipeline: page and paragraph control from the compiler's penalty, discretionary and parameter nodes — needs #568 re-pin (#64)#569
d-q222 wants to merge 1 commit into
mainfrom
agent/daniel-parent/page-para-control-pipeline

d-q222 commented Sep 15, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

d-q222 commented Sep 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Overlap check

Test results

Not done

Audit: page and paragraph control commands (GH-64)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

d-q222 commented Sep 15, 2026 •

edited

Loading