Conversation
What changed: - tests/fuzz_support/mod.rs: seeds from every .tex under fixtures/ and crates/*/tests; per-case splitmix RNG (seed, index) so every finding replays; mutations: truncate, delete/duplicate brace/bracket/$/\begin/\end tokens, splice lines between files, 10k-deep nesting, 100k-char control sequence names, unbalanced \if/\fi, \def recursion, invalid \char code points, \input of itself. Each case runs compile_full_project (parse + layout) on an 8 MiB thread under catch_unwind with a watchdog; worker processes isolate stack overflows and hangs; ddmin minimiser. - examples/fuzz_compile.rs: supervisor, --replay, --check, --minimise, --diagnostics. - tests/fuzz_compile.rs: the same run as an #[ignore] test (debug build). What was run: - fuzz_compile --cases 300 --jobs 12: "fuzz: 300 cases, 4 unique findings" - FLASHTEX_FUZZ_CASES=40 cargo test --test fuzz_compile -- --ignored: "the fuzzer found 3 unique failures" (expected before the fixes) Next step: 50k-case run, minimise and fix each finding. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
… u8 overflow What changed: - The per-kind and total list depths were `count() as u8 + 1` in push_list_frame and open_list: 255 enclosing lists panicked with "attempt to add with overflow" (fuzz finding). Depths now saturate (next_list_depths), and a \begin past LaTeX's \@toodeep limits (six \list levels; four itemize/enumerate) reports "LaTeX Error: Too deeply nested." while the list is still typeset. - enumitem start=<i64::MIN> no longer overflows `n - 1`. - Regression test in tests/robustness.rs. What was run: - cargo test --test robustness lists_nested (before the fix): "panicked at src/parser.rs:6972:26: attempt to add with overflow" - after: "test result: ok. 1 passed; 0 failed" Next step: remaining fuzz findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
What changed:
- finish_math took an unclosed formula's end from the last content token.
When the content was re-read from earlier input (`\setlength{\begin{}$`)
that token lay before the `$`, so Span::in_document tripped
"span start must not exceed end" (debug) or produced an inverted span
(release). The end now comes only from the opener's document and never
precedes the opener's end.
- Regression test an_unclosed_math_span_never_inverts.
What was run:
- before: "panicked at src/lib.rs:60:9: span start must not exceed end"
- after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed"
Next step: the cross-document alignment span.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
… in one document
What changed:
- multirow_environment merged every cell token's span into the row span;
with `\begin{align}a\include{sub}` the cells hold tokens of sub.tex and
main.tex, tripping "cannot merge spans from different documents". Row
spans now merge only tokens of the environment's own document, and an
`\end{align}` found in another document no longer sets the display's
end offset (it could invert the span).
- Regression test an_alignment_that_inputs_another_document_keeps_its_spans_in_one_document.
What was run:
- before: "panicked at src/lib.rs:74:9: assertion `left == right` failed:
cannot merge spans from different documents"
- after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed"
Next step: remaining fuzz findings (tex-expansion overflow, stack overflow, hangs).
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…flow What changed: - Fuzz finding: `\advance` added to a count holding i64::MAX (an over-long digit string parsed to i64::MAX) and panicked with "attempt to add with overflow" (expand.rs do_arith). - Constants follow tex.web §445: past 2^31-1 is "Number too big." and becomes 2147483647 (decimal, octal, hex; also the integer part of a dimen/glue, which silently became 0 before). - `<dimen>` past max_dimen is "Dimension too large." and becomes ±max_dimen (§448). - \advance and \multiply check the result against infinity / max_dimen; on overflow they report "Arithmetic overflow." and assign nothing (§1236-1240). - \numexpr/\dimexpr: saturating steps, and an out-of-range result is "Arithmetic overflow." with value 0 (e-TeX); rounded_div computes in i128. - `fil` followed by more than two further `l`s overflowed the u8 order; now "Illegal unit of measure (replaced by filll)." (§454). - Regression test numeric_ranges_follow_tex_instead_of_overflowing. What was run: - before: left "9223372036854775807" (and the add-overflow panic) - after: cargo test (tex-expansion): "test result: ok. 56 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: stack overflow and hang findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…e stack
What changed:
- Fuzz finding: a table cell, footnote, \colorbox/\rotatebox/\uline
argument or \include re-enters parse_stream on its own token stream
with no depth bound. 3000 nested tabulars overflowed an 8 MiB release
stack ("thread has overflowed its stack"), and 10k took minutes (every
level copies its cell tokens).
- parse_stream now counts its nesting; past STREAM_DEPTH_LIMIT = 255 it
reports "TeX capacity exceeded, sorry [grouping levels=255]." once per
compile and skips the content nested past the limit.
- fuzz_support: FLASHTEX_FUZZ_STACK_KB overrides the case stack size.
- Regression test nested_sub_parses_hit_tex_grouping_capacity_instead_of_the_stack
(debug frames are ~50 KiB per tabular level, so it runs on a 64 MiB thread).
What was run:
- before: "thread '<unknown>' has overflowed its stack / fatal runtime error"
- after: "test result: ok. 1 passed; 0 failed" (5.86s, debug)
- release --check at depth 255, 2 MiB stack: "signature: ok" (tabular, footnote)
Next step: hangs (\loop runaway, unknown-command suggestion cost).
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…x.web §339)
What changed:
- Fuzz finding (hang): `\loop{\n}` with no \repeat. The file ended while
\loop's argument was scanned; the expander reported the runaway but still
expanded the body with the partial argument, so \iterate ran until the
2M expansion-step limit, emitting "Extra \fi." (and, via the compiler,
one unknown-command diagnostic) per iteration: >2 minutes to compile.
- TeX sets long_state := outer_call there, so the inserted \par aborts the
call. report_file_ended_and_recover now sets the silent runaway flag
for Matching, and call_macro aborts.
- Regression test a_macro_call_cut_off_by_the_end_of_file_is_aborted.
What was run:
- before: diagnostics ["Runaway argument?...", "Extra \fi.", "Extra \fi.", ...]
- after: cargo test (tex-expansion): "ok. 57 passed; 0 failed" (expand_tests),
"ok. 3 passed" (incremental), "ok. 2 passed" (oracle)
Next step: the unknown-command suggestion cost behind the remaining hangs.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
… cannot hang
What changed:
- Fuzz finding (hang): `\def\a{\n\a}\a` emits ~666k `\n` before the 2M
expansion-step limit. Each got an unknown-command diagnostic whose
did-you-mean scanned the whole vocabulary several times
(command_error -> closest_commands, command_help -> suggest_command,
is_known_command), allocating per candidate: 269 s in release.
- closest_commands is memoised per thread (bounded at 4096 names);
is_known_command is a HashSet lookup; candidates whose length differs by
more than the edit limit are skipped before edit_distance copies the name
(a 100k-character control sequence: 860 ms -> 6 ms).
- Regression test a_runaway_loop_of_unknown_commands_is_diagnosed_in_bounded_time.
What was run:
- release, before: "269114 ms, 1 pages, 666816 diagnostics"
- release, after: "1989 ms, 1 pages, 666816 diagnostics"
- debug test after: "test result: ok. 1 passed; 0 failed ... finished in 10.88s"
Next step: re-run the 50k fuzz on the fixed build.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…ty exceeded"
What changed:
- Fuzz finding (hang, 9 of 26 hangs in a 2000-case run):
`\def\a#1{\a{#1#1}}\a x` doubles its argument on every call, so the
token lists exhaust memory and time after ~30 calls while the 2M
expansion-step limit is nowhere near.
- call_macro sizes the substitution before building it; past the
output-token budget (Limits::max_output_tokens) it reports
"TeX capacity exceeded, sorry [main memory size=5000000]." and stops
expansion, as TeX does when main memory runs out.
- Regression test an_argument_that_doubles_every_call_exceeds_capacity.
What was run:
- before (release fuzz build, --check, 40 s watchdog): cases 1204 1196 1599
1849 567 894 393 941 1280 "signature: hang"
- after: all nine "signature: ok" in <= 1 s
- cargo test (tex-expansion): "ok. 58 passed; 0 failed" (expand_tests),
"ok. 3 passed" (incremental), "ok. 2 passed" (oracle)
Next step: the slow deep-nesting cases.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
What changed:
- Reported from another lane: parsing
crates/page-builder/oracle/fixtures/vspace-03.tex panicked with
"span start must not exceed end" in required_group_bounded. Minimised:
`\AtBeginDocument{\setlength\}}` + `\begin{document`. The replayed
argument's closing offset lies before its `{`.
- One helper, P::span_through(open, end), now builds every
"opener through end offset" span (required_group_bounded, display math,
alignments, tabular, the delimited-argument reader): the end never
precedes the opener's end or runs past the opener's document.
- Regression test an_argument_replayed_from_at_begin_document_never_inverts_its_span
(minimised input plus the whole fixture).
What was run:
- before: "panicked at src/lib.rs:60:9: span start must not exceed end"
- after: "test result: ok. 1 passed; 0 failed"
Next step: \loop oracle fixtures parse time; slow deep-nesting cases.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…rash inputs What changed: - Seeds now include crates/*/oracle/**/*.tex (the page-builder oracle fixture vspace-03.tex panicked the parser and was not a seed). This changes which input each case index generates. - Up to 50 inputs per hang/crash signature are saved for triage (those signatures do not distinguish causes). What was run: - fuzz_compile --cases 2000: "fuzz: 2000 cases, 1 unique findings" (26 hangs, triaged) Next step: full 50k re-run on the fixed build. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…bug threads) What changed: - Measured after the previous commit: 255 nested tabulars still overflow a 512 KiB thread in release (128 already did; ~7 KiB a level) and a 2 MiB debug thread fails at 40 levels (~50 KiB a level), so the guard did not fire first on every profile. STREAM_DEPTH_LIMIT is now 32, like MAX_MATH_DEPTH; the error reads "TeX capacity exceeded, sorry [grouping levels=32]." - \include nesting keeps its own INCLUDE_DEPTH_LIMIT (64) and no longer counts toward the sub-parse depth. - The regression test runs on the default test thread again. What was run: - release fuzz build, FLASHTEX_FUZZ_STACK_KB=512, 3000 nested tabular / footnote: "signature: ok" (both) - cargo test --test robustness nested_sub_parses: "test result: ok. 1 passed" (0.28s) Next step: \left...\right nesting is quadratic in layout. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…not quadratic layout What changed: - Fuzz finding (slow compile, 15.7 s release for one input): sizing a \left/\right pair lays out all the atoms it encloses (left_right_stretch_scales), so 10k nested pairs cost depth x length. - The math parser counts open \left's; past MAX_LEFT_RIGHT_DEPTH = 32 it reports "TeX capacity exceeded, sorry [grouping levels=32]." once and drops the extra \left's and their \right's. Formulas within the limit are unchanged (left_right_nesting goldens pass). - Regression test deeply_nested_left_right_pairs_hit_a_capacity_limit_in_bounded_time. What was run: - before: fuzz case 529 "15664 ms, 2 pages, 17120 diagnostics" (release) - after: debug test with 10k nested pairs "ok. 1 passed ... finished in 0.13s" - cargo test --test left_right_nesting: "test result: ok. 3 passed; 0 failed" Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…ot define
What changed:
- self_referential_macro_hits_explicit_recursion_limit used
`\newcommand{\loop}{\loop} \loop`. The prelude already defines \loop, so
\newcommand keeps `\loop#1\repeat`, whose argument runs to the end of
the file. It only reached the step limit through the bug fixed in
"tex-expansion: a macro call cut off by the end of file is aborted"; TeX
reports the runaway argument instead. The test now uses \recurse.
What was run:
- cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast:
58 suites, 676 passed, 0 failed, 6 ignored
Next step: 50k fuzz re-run, PR.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
What changed: - rustfmt on the new fuzz files and tests/robustness.rs (which was rustfmt-clean on main). No behaviour change. What was run: - cargo test --test robustness --test fuzz_compile: "test result: ok. 13 passed" and "ok. 0 passed; 0 failed; 1 ignored" Next step: PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…city exceeded"
What changed:
- Fuzz finding (36 GB resident, killed): `\def\a{\csname a\endcsna` +
10k `[` ... `]` + `me}\a`. `\csname a` re-enters \a before the rest
of its 20k-token body is read, so every call adds an input level holding
that body; memory ran out long before the 2M expansion-step limit.
- Pushing a token list now checks TeX's limits: 10000 input levels
("TeX capacity exceeded, sorry [input stack size=10000].", TeX Live's
stack_size) and, every 64 levels, the tokens still pending across all
levels against the output-token budget ("TeX capacity exceeded, sorry
[main memory size=5000000]."). Either stops expansion.
- Regression test a_non_tail_self_call_exceeds_capacity_instead_of_memory.
What was run:
- before: /usr/bin/time -l fuzz_compile --check case-14361: "36046176256
maximum resident set size", 80 s, killed (exit 137) in a second run
- after: "30 ms, 1 pages, 133 diagnostics", "172441600 maximum resident set size"
- cargo test (tex-expansion): "ok. 59 passed; 0 failed" (expand_tests),
"ok. 3 passed" (incremental), "ok. 2 passed" (oracle)
Next step: remaining slow cases; full compiler suite; PR.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…ry block What changed: - Fuzz finding (25.7 GB resident): 30k nested itemize. Every block clones the enclosing list frames, so memory was blocks x depth. - push_list_frame now stores no frame past LaTeX's limit (the one that reports "LaTeX Error: Too deeply nested."); it counts dropped levels, and a list \end pops a dropped level first. Blocks hold at most six frames; the dropped levels typeset at the deepest stored level. - lists_nested_past_255_levels_are_too_deeply_nested_not_an_overflow also checks that no block stores more than six frames. What was run: - before: fuzz case 12106 "4117 ms, 884 pages", "25687048192 maximum resident set size"; test: "a block stored 7 list frames" - after: "622 ms, 884 pages", "138084352 maximum resident set size"; test "ok. 1 passed" - cargo test --test list_structure --test setlist_spacing --test acceptance --test pinned_fixtures --lib: all "test result: ok" Next step: remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
What changed: - Fuzz finding (22 s): a mutated microtype oracle fixture whose \loop never ends. do_conditional (and a taken branch's \else) called line_of_span on every \if, counting newlines from the start of the file, for the "Incomplete \if...; all text was ignored after line N." message that only an unterminated conditional prints: quadratic in a runaway loop. - ScannerStatus::Skipping now carries the \if's span; the line is counted when that error is reported. The unused line count on a mismatched \end is gone. - Regression test conditionals_in_a_runaway_loop_do_not_rescan_the_source (also pins the "after line 3." wording). What was run: - before: test panicked "244.136504125s" (debug, 30 s budget) - after: "test result: ok. 1 passed ... finished in 0.53s"; tex-expansion cargo test: "ok. 60 passed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
What changed:
- The new tex-expansion capacity stops ("TeX capacity exceeded, sorry
[input stack size=10000]." / "[main memory size=5000000].") end
expansion at a point that depends on where the run started. The
expansion cache only knew the step limit, so an incremental run could
stop elsewhere than a full one: expansion_incremental's random-edit test
diverged at step 198 ("tokens differ at 60356 (cached 60356 vs full
60386 tokens)"). The rest of the document was also dropped instead of
typeset unexpanded, and the recovery note said "continued expanding".
- step_limit_hit (cache fallback, unexpanded resume) and recovery_for now
treat both stops alike.
- Regression test a_capacity_stop_is_recovered_like_the_step_limit.
What was run:
- before: expansion_incremental FAILED as above; robustness test: left
Some("continued expanding after the problem")
- after: cargo test --test expansion_incremental: "ok. 2 passed; 0 failed";
cargo test --test robustness a_capacity_stop: "ok. 1 passed"
Next step: full compiler suite, final fuzz run, PR.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
Contributor
Author
|
daniel-parent independent review (subagent) Verdict: CHANGES NEEDED (one confirmed correctness bug; everything else checks out and is well-evidenced — see below) Reviewed by building 1. Depth/capacity limits vs. real TeX
2. Real TeX's
3. Everything else checks out, verified directly (not just read):
4. Fixture-wide diagnostic sweep
Nice piece of work overall — the TeX-parameter research (texmf.cnf constants, |
…no diagnostic What changed: - `\advance` on count, dimen and skip registers (value, stretch and shrink) adds with explicit i32 wrapping (tex.web §1238-1239 has no range check) instead of reporting "Arithmetic overflow." and leaving the register. `\multiply`, `\divide`, `\numexpr` and `\dimexpr` keep their bounds checks. - numeric_ranges_follow_tex_instead_of_overflowing pins pdfTeX's wrapped values with no diagnostic. What was run: - pdftex 3.141592653-2.6-1.40.29 (TeX Live 2026), plain and -etex: \count0=2147483647 \advance\count0 by 1 \showthe\count0 -> -2147483648 \count0=-2147483647 \advance\count0 by -2 -> 2147483647 \dimen0=\maxdimen \advance\dimen0 by 1sp \showthe\dimen0 -> 16384.0pt \dimen0=16383pt \advance\dimen0 by 16383pt -> 32766.0pt 3x\maxdimen -> -16384.00005pt; 4x\maxdimen+1sp -> -0.00005pt skip \maxdimen plus \maxdimen minus 1pt, advanced by the same minus 2pt -> 32767.99997pt plus 32767.99997pt minus 3.0pt - cargo test -p flashtex-tex-expansion --test expand_tests: test result: ok. 60 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out Next step: reword or raise the grouping-level capacity limit (#577 review). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
…s where pdflatex does
What changed:
- STREAM_DEPTH_LIMIT stays 32 (255 nested tabulars, footnotes or rotateboxes
overflow a 512 KiB release and a 2 MiB debug thread), and its error now
says "FlashTeX nesting limit (32) exceeded" instead of claiming TeX's
"grouping levels" capacity.
- MAX_LEFT_RIGHT_DEPTH rises from 32 to 253 (TEX_GROUPING_LEVELS - 2), with
"TeX capacity exceeded, sorry [grouping levels=255].": pdflatex accepts 253
nested \left in a document's $$ or $ math and stops at the 254th.
The \left counter is not recursive, so the stack is not the bound.
What was run:
- pdflatex (TeX Live 2026), \documentclass{article}\begin{document}$$...$$:
252 ok, 253 ok, 254 and 255 "! TeX capacity exceeded, sorry [grouping levels=255]."
inline $...$: 253 ok, 254 capacity exceeded.
- Scratch probe (not committed) with both limits at 255:
release, 512 KiB thread: tabular/footnote/rotatebox 255 -> stack overflow;
colorbox/uline 255 ok; \left 255 ok (also on 256 KiB), 10000 in 120 ms
debug, 2 MiB thread: tabular/footnote/colorbox/rotatebox/uline 255 -> stack overflow;
\left 255 ok (also on 512 KiB), 10000 in 714 ms
- cargo test -p flashtex-compiler --test robustness:
test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 36.36s
Next step: full compiler and tex-expansion suites and a 5k fuzz smoke run.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
…flood-cap What changed: merged origin/agent/daniel-parent/compiler-fuzz d5d308e (no rebase), per the coordinator. What was run: full compiler and tex-expansion suites on the merge (see the PR). Next step: open the PR stacked on #577. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
Contributor
Author
|
daniel-parent (subagent): review findings fixed in 0c1524c and d5d308e. 1.
2. Grouping limit.
Test results
The PR body's rows 4, 5 and 10 are updated. Unrelated, not changed: FlashTeX reads |
Contributor
Author
|
GH-DIAG-FLOOD-CAP: merge this branch in. daniel-parent (subagent) pushed the #577 review fixes to agent/daniel-parent/compiler-fuzz (0c1524c, d5d308e). The diag-flood-cap lane (agent/daniel-parent/diag-flood-cap, stacked on #577) should merge it in with a normal merge, not a force push. Note that the sub-parse depth message is now |
This was referenced Sep 15, 2026
Merged
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
What changed:
- tabular_cell_inlines and box_inlines parse their content as blocks and
fold them into inlines. They dropped those blocks' block_dependencies
entries but not their block_par_leading entries, so Parsed carried more
leadings than blocks and every later paragraph's \baselineskip was
read from the wrong block. Both now truncate block_par_leading too, as
argument_inlines already did.
- Fuzz finding (render-pipeline fuzz_render): minimised input
`\usepackage{longtable\begin{longtable}\begin{longtable` panicked
`adapter.rs:1261 assertion left == right failed` (the pipeline's
debug_assert_eq on the two lengths). Still present on #577's branch.
What was run:
- cargo test --test block_par_leading, before: test result: FAILED. 0
passed; 3 failed; after: test result: ok. 3 passed; 0 failed
- cargo test --lib: ok. 339 passed; --test par_leading: ok. 7 passed;
xcolor_parse ok. 8; siunitx_columns ok. 5; letter_class ok. 11;
kernel_untested_a ok. 20 passed, 1 ignored
Next step: re-pin vendor/compiler in render-pipeline (owner lane).
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
What changed: - `\addbibresource` and `\printbibliography` move out of `P::command` into `#[inline(never)]` helpers (`add_bib_resource`, `print_bibliography`); `biblatex_cite` is `#[inline(never)]` too. - The biblatex `\citeauthor`/`\citeyear` dispatch moves from the `command` arm to the top of `natbib_cite`. - Why: `P::command` is on the stack once per nested sub-parse. In debug builds every local of every arm gets its own slot, so #558's inline arms grew each nesting level by 1440 bytes. Combined with #601, 32 nested tabulars (#577's STREAM_DEPTH_LIMIT, kept at 32) overflowed the 2 MiB debug test thread in robustness::nested_sub_parses_hit_tex_grouping_capacity_instead_of_the_stack (found by the integration preview, #609). What was run: - Stack per nesting level (debug), measured with a scratch probe that prints the stack address in `parse_stream` at each depth (not committed): | construct | #601 | #601+#558 before | #601+#558 after | | tabular | 57232 | 58672 | 57360 | | footnote | 31424 | 32864 | 31552 | | colorbox | 30928 | 32368 | 31056 | | rotatebox | 32048 | 33488 | 32176 | | uline | 30336 | 31776 | 30464 | - #601 + #558 (merge), debug, CARGO_BUILD_JOBS=4: cargo test --test robustness --test biblatex biblatex: test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s robustness: test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 41.06s - This branch, debug: biblatex: test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s robustness: test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 40.81s natbib: test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s --lib cite: test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 331 filtered out; finished in 0.00s Next step: merge into the integration preview (#609) and rerun there. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
This was referenced Sep 15, 2026
Merged
Closed
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
What changed: - adapter::list_stack_at and in_theorem_environment searched `source[from..at]` for the next \begin and the next \end again after every hit; whenever one was absent that search ran to `at`, so each call was O(begins x bytes). Both now replay one ordered begin_end_commands scan, found exactly as find_command finds them. - That scan and setlist_calls' \setlist ranges are indexed once per text for the adapt call, on the existing MacroDefsScope entry (the same (address, length) key the macro-definition index uses), since they run for every list item and every paragraph. - Fuzz findings (render-pipeline fuzz_render, "hang in render" with the compiler fixes of #577 applied): a mutated enumitem worksheet 37.5 s -> 0.29 s; page-builder enlarge-00 21.5 s -> 0.19 s; a 715-page list of 20 000 items 23.3 s -> 0.13 s; 1 000 nested itemize 23.8 s -> 0.15 s; a plain 5 000-paragraph article with a center per paragraph 8.0 s -> 1.1 s; 300 items inside 1 000 open centers 7.4 s -> 25 ms (identical PDF bytes). All release timings through fuzz_render --diagnostics. What was run: - cargo test --test fuzz_regressions -- a_long_list list_items_inside, before: FAILED. 0 passed; 2 failed (17.3 s and 18.7 s to adapt); after: test result: ok. 2 passed; 0 failed; finished in 0.27s - cargo test --no-fail-fast (render-pipeline): 368 passed, 12 failed; all 12 panic at src/adapter.rs:1261 (block_par_leading debug assert, #517) in letter_class_geometry, uline, underline_sout, xcolor_oracle; letter_class_geometry fails identically with this change stashed. Next step: the 50k rerun on the fixed branch. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
What changed:
- typeset::merge_spans_in merges only the spans in the construct's own
document (falling back to its span). rows_block merged every atom's
span of an amsmath cell, and fenced merged \left with \right; an
\include inside the construct puts atoms in another document, and
Span::merge asserts one document (release builds silently mixed the
offsets of two files).
- Fuzz finding: `\begin{align}e\include{sub}` with sub.tex
`x\input{sub}` (minimised from 507 bytes). On this branch the vendored
compiler panics first (#577 bug 3); with #577's compiler the pipeline
then panicked `cannot merge spans from different documents` in
Context::rows_block (RUST_BACKTRACE). With this change the #577-compiler
build renders it: fuzz_render --check: signature: ok.
What was run:
- cargo test --lib merge_spans_in: test result: ok. 1 passed; 0 failed
- a scratch build of this crate against #577's compiler (not committed):
before: panic vendor/compiler/src/lib.rs:74:9 in render; after: ok
Next step: final 50k runs.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
d-q222
added a commit
that referenced
this pull request
Sep 15, 2026
… runs What changed: - MathRec::text_runs is an Rc<[TextRun]>. Context::math_pieces clones the MathRec for every piece after the first, which copied every \text run each time: memory grew as break points x runs. - tests/math_break_memory.rs measures the peak heap of one render with a counting global allocator (std only). - Fuzz finding (render-pipeline fuzz_render with #577's compiler, "memory over 4096 MiB in render"): a mutated amsmath fixture with an overfull `$x+x+...$` around 10k nested \textbf passed 3.4 GB within a second of the render stage. Synthetic: 1000 `x+` either side of 1000 nested \textbf peaked over 3 GB; with FLASHTEX_INLINE_MATH_BREAKS=0 it stayed small. What was run: - cargo test --test math_break_memory, before: FAILED (rendering peaked at 1250 MiB of heap); after: test result: ok. 1 passed; 0 failed - inline_math_breaks ok. 1; math_text ok. 2; v2_and_math ok. 7; display_math ok. 4; math_glyph_spans ok. 6 Next step: final 50k runs. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
GoKubar
added a commit
that referenced
this pull request
Sep 15, 2026
…ile (stacked on #601) (#610) * compiler: deterministic mutation fuzzer (stable Rust, no new deps) What changed: - tests/fuzz_support/mod.rs: seeds from every .tex under fixtures/ and crates/*/tests; per-case splitmix RNG (seed, index) so every finding replays; mutations: truncate, delete/duplicate brace/bracket/$/\begin/\end tokens, splice lines between files, 10k-deep nesting, 100k-char control sequence names, unbalanced \if/\fi, \def recursion, invalid \char code points, \input of itself. Each case runs compile_full_project (parse + layout) on an 8 MiB thread under catch_unwind with a watchdog; worker processes isolate stack overflows and hangs; ddmin minimiser. - examples/fuzz_compile.rs: supervisor, --replay, --check, --minimise, --diagnostics. - tests/fuzz_compile.rs: the same run as an #[ignore] test (debug build). What was run: - fuzz_compile --cases 300 --jobs 12: "fuzz: 300 cases, 4 unique findings" - FLASHTEX_FUZZ_CASES=40 cargo test --test fuzz_compile -- --ignored: "the fuzzer found 3 unique failures" (expected before the fixes) Next step: 50k-case run, minimise and fix each finding. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: lists nested past 255 levels are "Too deeply nested", not a u8 overflow What changed: - The per-kind and total list depths were `count() as u8 + 1` in push_list_frame and open_list: 255 enclosing lists panicked with "attempt to add with overflow" (fuzz finding). Depths now saturate (next_list_depths), and a \begin past LaTeX's \@toodeep limits (six \list levels; four itemize/enumerate) reports "LaTeX Error: Too deeply nested." while the list is still typeset. - enumitem start=<i64::MIN> no longer overflows `n - 1`. - Regression test in tests/robustness.rs. What was run: - cargo test --test robustness lists_nested (before the fix): "panicked at src/parser.rs:6972:26: attempt to add with overflow" - after: "test result: ok. 1 passed; 0 failed" Next step: remaining fuzz findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: an unclosed math span never ends before its opener What changed: - finish_math took an unclosed formula's end from the last content token. When the content was re-read from earlier input (`\setlength{\begin{}$`) that token lay before the `$`, so Span::in_document tripped "span start must not exceed end" (debug) or produced an inverted span (release). The end now comes only from the opener's document and never precedes the opener's end. - Regression test an_unclosed_math_span_never_inverts. What was run: - before: "panicked at src/lib.rs:60:9: span start must not exceed end" - after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed" Next step: the cross-document alignment span. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: an alignment that \input's another document keeps its spans in one document What changed: - multirow_environment merged every cell token's span into the row span; with `\begin{align}a\include{sub}` the cells hold tokens of sub.tex and main.tex, tripping "cannot merge spans from different documents". Row spans now merge only tokens of the environment's own document, and an `\end{align}` found in another document no longer sets the display's end offset (it could invert the span). - Regression test an_alignment_that_inputs_another_document_keeps_its_spans_in_one_document. What was run: - before: "panicked at src/lib.rs:74:9: assertion `left == right` failed: cannot merge spans from different documents" - after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed" Next step: remaining fuzz findings (tex-expansion overflow, stack overflow, hangs). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: TeX's integer and dimension ranges instead of i64 overflow What changed: - Fuzz finding: `\advance` added to a count holding i64::MAX (an over-long digit string parsed to i64::MAX) and panicked with "attempt to add with overflow" (expand.rs do_arith). - Constants follow tex.web §445: past 2^31-1 is "Number too big." and becomes 2147483647 (decimal, octal, hex; also the integer part of a dimen/glue, which silently became 0 before). - `<dimen>` past max_dimen is "Dimension too large." and becomes ±max_dimen (§448). - \advance and \multiply check the result against infinity / max_dimen; on overflow they report "Arithmetic overflow." and assign nothing (§1236-1240). - \numexpr/\dimexpr: saturating steps, and an out-of-range result is "Arithmetic overflow." with value 0 (e-TeX); rounded_div computes in i128. - `fil` followed by more than two further `l`s overflowed the u8 order; now "Illegal unit of measure (replaced by filll)." (§454). - Regression test numeric_ranges_follow_tex_instead_of_overflowing. What was run: - before: left "9223372036854775807" (and the add-overflow panic) - after: cargo test (tex-expansion): "test result: ok. 56 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: stack overflow and hang findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: nested sub-parses stop at TeX's 255 grouping levels, not the stack What changed: - Fuzz finding: a table cell, footnote, \colorbox/\rotatebox/\uline argument or \include re-enters parse_stream on its own token stream with no depth bound. 3000 nested tabulars overflowed an 8 MiB release stack ("thread has overflowed its stack"), and 10k took minutes (every level copies its cell tokens). - parse_stream now counts its nesting; past STREAM_DEPTH_LIMIT = 255 it reports "TeX capacity exceeded, sorry [grouping levels=255]." once per compile and skips the content nested past the limit. - fuzz_support: FLASHTEX_FUZZ_STACK_KB overrides the case stack size. - Regression test nested_sub_parses_hit_tex_grouping_capacity_instead_of_the_stack (debug frames are ~50 KiB per tabular level, so it runs on a 64 MiB thread). What was run: - before: "thread '<unknown>' has overflowed its stack / fatal runtime error" - after: "test result: ok. 1 passed; 0 failed" (5.86s, debug) - release --check at depth 255, 2 MiB stack: "signature: ok" (tabular, footnote) Next step: hangs (\loop runaway, unknown-command suggestion cost). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: a macro call cut off by the end of file is aborted (tex.web §339) What changed: - Fuzz finding (hang): `\loop{\n}` with no \repeat. The file ended while \loop's argument was scanned; the expander reported the runaway but still expanded the body with the partial argument, so \iterate ran until the 2M expansion-step limit, emitting "Extra \fi." (and, via the compiler, one unknown-command diagnostic) per iteration: >2 minutes to compile. - TeX sets long_state := outer_call there, so the inserted \par aborts the call. report_file_ended_and_recover now sets the silent runaway flag for Matching, and call_macro aborts. - Regression test a_macro_call_cut_off_by_the_end_of_file_is_aborted. What was run: - before: diagnostics ["Runaway argument?...", "Extra \fi.", "Extra \fi.", ...] - after: cargo test (tex-expansion): "ok. 57 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: the unknown-command suggestion cost behind the remaining hangs. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: unknown-command suggestions are memoised, so a runaway loop cannot hang What changed: - Fuzz finding (hang): `\def\a{\n\a}\a` emits ~666k `\n` before the 2M expansion-step limit. Each got an unknown-command diagnostic whose did-you-mean scanned the whole vocabulary several times (command_error -> closest_commands, command_help -> suggest_command, is_known_command), allocating per candidate: 269 s in release. - closest_commands is memoised per thread (bounded at 4096 names); is_known_command is a HashSet lookup; candidates whose length differs by more than the edit limit are skipped before edit_distance copies the name (a 100k-character control sequence: 860 ms -> 6 ms). - Regression test a_runaway_loop_of_unknown_commands_is_diagnosed_in_bounded_time. What was run: - release, before: "269114 ms, 1 pages, 666816 diagnostics" - release, after: "1989 ms, 1 pages, 666816 diagnostics" - debug test after: "test result: ok. 1 passed; 0 failed ... finished in 10.88s" Next step: re-run the 50k fuzz on the fixed build. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: a macro expansion past the token budget is "TeX capacity exceeded" What changed: - Fuzz finding (hang, 9 of 26 hangs in a 2000-case run): `\def\a#1{\a{#1#1}}\a x` doubles its argument on every call, so the token lists exhaust memory and time after ~30 calls while the 2M expansion-step limit is nowhere near. - call_macro sizes the substitution before building it; past the output-token budget (Limits::max_output_tokens) it reports "TeX capacity exceeded, sorry [main memory size=5000000]." and stops expansion, as TeX does when main memory runs out. - Regression test an_argument_that_doubles_every_call_exceeds_capacity. What was run: - before (release fuzz build, --check, 40 s watchdog): cases 1204 1196 1599 1849 567 894 393 941 1280 "signature: hang" - after: all nine "signature: ok" in <= 1 s - cargo test (tex-expansion): "ok. 58 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: the slow deep-nesting cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: argument and display spans never invert on replayed tokens What changed: - Reported from another lane: parsing crates/page-builder/oracle/fixtures/vspace-03.tex panicked with "span start must not exceed end" in required_group_bounded. Minimised: `\AtBeginDocument{\setlength\}}` + `\begin{document`. The replayed argument's closing offset lies before its `{`. - One helper, P::span_through(open, end), now builds every "opener through end offset" span (required_group_bounded, display math, alignments, tabular, the delimited-argument reader): the end never precedes the opener's end or runs past the opener's document. - Regression test an_argument_replayed_from_at_begin_document_never_inverts_its_span (minimised input plus the whole fixture). What was run: - before: "panicked at src/lib.rs:60:9: span start must not exceed end" - after: "test result: ok. 1 passed; 0 failed" Next step: \loop oracle fixtures parse time; slow deep-nesting cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler fuzz: seed from crates/*/oracle too; keep a sample of hang/crash inputs What changed: - Seeds now include crates/*/oracle/**/*.tex (the page-builder oracle fixture vspace-03.tex panicked the parser and was not a seed). This changes which input each case index generates. - Up to 50 inputs per hang/crash signature are saved for triage (those signatures do not distinguish causes). What was run: - fuzz_compile --cases 2000: "fuzz: 2000 cases, 1 unique findings" (26 hangs, triaged) Next step: full 50k re-run on the fixed build. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: sub-parse depth limit 32 (fits 512 KiB release and 2 MiB debug threads) What changed: - Measured after the previous commit: 255 nested tabulars still overflow a 512 KiB thread in release (128 already did; ~7 KiB a level) and a 2 MiB debug thread fails at 40 levels (~50 KiB a level), so the guard did not fire first on every profile. STREAM_DEPTH_LIMIT is now 32, like MAX_MATH_DEPTH; the error reads "TeX capacity exceeded, sorry [grouping levels=32]." - \include nesting keeps its own INCLUDE_DEPTH_LIMIT (64) and no longer counts toward the sub-parse depth. - The regression test runs on the default test thread again. What was run: - release fuzz build, FLASHTEX_FUZZ_STACK_KB=512, 3000 nested tabular / footnote: "signature: ok" (both) - cargo test --test robustness nested_sub_parses: "test result: ok. 1 passed" (0.28s) Next step: \left...\right nesting is quadratic in layout. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: \left...\right nesting past 32 levels is a capacity error, not quadratic layout What changed: - Fuzz finding (slow compile, 15.7 s release for one input): sizing a \left/\right pair lays out all the atoms it encloses (left_right_stretch_scales), so 10k nested pairs cost depth x length. - The math parser counts open \left's; past MAX_LEFT_RIGHT_DEPTH = 32 it reports "TeX capacity exceeded, sorry [grouping levels=32]." once and drops the extra \left's and their \right's. Formulas within the limit are unchanged (left_right_nesting goldens pass). - Regression test deeply_nested_left_right_pairs_hit_a_capacity_limit_in_bounded_time. What was run: - before: fuzz case 529 "15664 ms, 2 pages, 17120 diagnostics" (release) - after: debug test with 10k nested pairs "ok. 1 passed ... finished in 0.13s" - cargo test --test left_right_nesting: "test result: ok. 3 passed; 0 failed" Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: the recursion-limit unit test uses a name the kernel does not define What changed: - self_referential_macro_hits_explicit_recursion_limit used `\newcommand{\loop}{\loop} \loop`. The prelude already defines \loop, so \newcommand keeps `\loop#1\repeat`, whose argument runs to the end of the file. It only reached the step limit through the bug fixed in "tex-expansion: a macro call cut off by the end of file is aborted"; TeX reports the runaway argument instead. The test now uses \recurse. What was run: - cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast: 58 suites, 676 passed, 0 failed, 6 ignored Next step: 50k fuzz re-run, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler fuzz: rustfmt the harness and robustness tests What changed: - rustfmt on the new fuzz files and tests/robustness.rs (which was rustfmt-clean on main). No behaviour change. What was run: - cargo test --test robustness --test fuzz_compile: "test result: ok. 13 passed" and "ok. 0 passed; 0 failed; 1 ignored" Next step: PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: input stack and pending-token capacity, as TeX's "capacity exceeded" What changed: - Fuzz finding (36 GB resident, killed): `\def\a{\csname a\endcsna` + 10k `[` ... `]` + `me}\a`. `\csname a` re-enters \a before the rest of its 20k-token body is read, so every call adds an input level holding that body; memory ran out long before the 2M expansion-step limit. - Pushing a token list now checks TeX's limits: 10000 input levels ("TeX capacity exceeded, sorry [input stack size=10000].", TeX Live's stack_size) and, every 64 levels, the tokens still pending across all levels against the output-token budget ("TeX capacity exceeded, sorry [main memory size=5000000]."). Either stops expansion. - Regression test a_non_tail_self_call_exceeds_capacity_instead_of_memory. What was run: - before: /usr/bin/time -l fuzz_compile --check case-14361: "36046176256 maximum resident set size", 80 s, killed (exit 137) in a second run - after: "30 ms, 1 pages, 133 diagnostics", "172441600 maximum resident set size" - cargo test (tex-expansion): "ok. 59 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: remaining slow cases; full compiler suite; PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: list levels past \@toodeep are counted, not copied into every block What changed: - Fuzz finding (25.7 GB resident): 30k nested itemize. Every block clones the enclosing list frames, so memory was blocks x depth. - push_list_frame now stores no frame past LaTeX's limit (the one that reports "LaTeX Error: Too deeply nested."); it counts dropped levels, and a list \end pops a dropped level first. Blocks hold at most six frames; the dropped levels typeset at the deepest stored level. - lists_nested_past_255_levels_are_too_deeply_nested_not_an_overflow also checks that no block stores more than six frames. What was run: - before: fuzz case 12106 "4117 ms, 884 pages", "25687048192 maximum resident set size"; test: "a block stored 7 list frames" - after: "622 ms, 884 pages", "138084352 maximum resident set size"; test "ok. 1 passed" - cargo test --test list_structure --test setlist_spacing --test acceptance --test pinned_fixtures --lib: all "test result: ok" Next step: remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: an \if's line is computed only when its error is reported What changed: - Fuzz finding (22 s): a mutated microtype oracle fixture whose \loop never ends. do_conditional (and a taken branch's \else) called line_of_span on every \if, counting newlines from the start of the file, for the "Incomplete \if...; all text was ignored after line N." message that only an unterminated conditional prints: quadratic in a runaway loop. - ScannerStatus::Skipping now carries the \if's span; the line is counted when that error is reported. The unused line count on a mismatched \end is gone. - Regression test conditionals_in_a_runaway_loop_do_not_rescan_the_source (also pins the "after line 3." wording). What was run: - before: test panicked "244.136504125s" (debug, 30 s budget) - after: "test result: ok. 1 passed ... finished in 0.53s"; tex-expansion cargo test: "ok. 60 passed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: a TeX capacity stop is recovered like the expansion step limit What changed: - The new tex-expansion capacity stops ("TeX capacity exceeded, sorry [input stack size=10000]." / "[main memory size=5000000].") end expansion at a point that depends on where the run started. The expansion cache only knew the step limit, so an incremental run could stop elsewhere than a full one: expansion_incremental's random-edit test diverged at step 198 ("tokens differ at 60356 (cached 60356 vs full 60386 tokens)"). The rest of the document was also dropped instead of typeset unexpanded, and the recovery note said "continued expanding". - step_limit_hit (cache fallback, unexpanded resume) and recovery_for now treat both stops alike. - Regression test a_capacity_stop_is_recovered_like_the_step_limit. What was run: - before: expansion_incremental FAILED as above; robustness test: left Some("continued expanding after the problem") - after: cargo test --test expansion_incremental: "ok. 2 passed; 0 failed"; cargo test --test robustness a_capacity_stop: "ok. 1 passed" Next step: full compiler suite, final fuzz run, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler, tex-expansion: bound diagnostic floods from runaway loops What changed: - tex-expansion: an identical diagnostic (severity, message, span) is recorded once between safe points; the set is emptied when next_content_token returns at a safe point, which is where incremental checkpoints are taken, so restored engines decide exactly as a full run. - tex-expansion: "group nesting limit exceeded" / "conditional nesting limit exceeded" are reported once per excursion (flags in checkpointed State). - tex-expansion: \end compares @Currenvir part by part without rebuilding the name; environment names longer than 100 characters are shortened in messages only. - compiler: diagnostics::limit_repeats drops identical repeats and keeps 1000 distinct diagnostics per code, then one "further N similar diagnostics suppressed" summary per code; fatal stops are always kept. Applied to the parser's list and to the layout's list, identically in clean and incremental compiles. - compiler: unsupported / unsupported_preamble skip building a repeat of a command already reported at the same span. - fuzz harness: per-case diagnostic count and time, max diagnostics and slowest case in the report; example --diagnostics prints the most frequent messages; --digest hashes pages and diagnostics of every seed. - tests: diagnostic_floods.rs (child process per case, peak heap via a counting allocator, watchdog), engine excursion/dedupe/long-name tests, incremental equivalence with repeats and nesting limits, warm-vs-fresh JSON identity with bounded diagnostics. What was run: - cargo test --manifest-path crates/tex-expansion/Cargo.toml: test result: ok. 63 passed; 0 failed (expand_tests), test result: ok. 4 passed; 0 failed (incremental_tests), test result: ok. 2 passed; 0 failed (oracle_tests) - cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast: 59 suites, 684 passed, 0 failed, 6 ignored - fuzz_compile --cases 50000 --jobs 6 --seed 0xF1A57E40 (overflow checks, debug assertions): 0 panics, 0 overflows, 14 hangs (base 18); max diagnostics in one case 3109 (base 2974694) - fuzz_compile --digest over 419 seeds: pages identical for all 419; diagnostics differ in 49 (48 \loop fixtures ~14750 -> ~134, and 20-Vmatrix 3 -> 2, a duplicate glyph note) Next step: merge the updated compiler-fuzz branch and open the PR stacked on #577. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: \advance wraps in 32-bit arithmetic like pdfTeX, with no diagnostic What changed: - `\advance` on count, dimen and skip registers (value, stretch and shrink) adds with explicit i32 wrapping (tex.web §1238-1239 has no range check) instead of reporting "Arithmetic overflow." and leaving the register. `\multiply`, `\divide`, `\numexpr` and `\dimexpr` keep their bounds checks. - numeric_ranges_follow_tex_instead_of_overflowing pins pdfTeX's wrapped values with no diagnostic. What was run: - pdftex 3.141592653-2.6-1.40.29 (TeX Live 2026), plain and -etex: \count0=2147483647 \advance\count0 by 1 \showthe\count0 -> -2147483648 \count0=-2147483647 \advance\count0 by -2 -> 2147483647 \dimen0=\maxdimen \advance\dimen0 by 1sp \showthe\dimen0 -> 16384.0pt \dimen0=16383pt \advance\dimen0 by 16383pt -> 32766.0pt 3x\maxdimen -> -16384.00005pt; 4x\maxdimen+1sp -> -0.00005pt skip \maxdimen plus \maxdimen minus 1pt, advanced by the same minus 2pt -> 32767.99997pt plus 32767.99997pt minus 3.0pt - cargo test -p flashtex-tex-expansion --test expand_tests: test result: ok. 60 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out Next step: reword or raise the grouping-level capacity limit (#577 review). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: the sub-parse depth limit is FlashTeX's; \left nesting stops where pdflatex does What changed: - STREAM_DEPTH_LIMIT stays 32 (255 nested tabulars, footnotes or rotateboxes overflow a 512 KiB release and a 2 MiB debug thread), and its error now says "FlashTeX nesting limit (32) exceeded" instead of claiming TeX's "grouping levels" capacity. - MAX_LEFT_RIGHT_DEPTH rises from 32 to 253 (TEX_GROUPING_LEVELS - 2), with "TeX capacity exceeded, sorry [grouping levels=255].": pdflatex accepts 253 nested \left in a document's $$ or $ math and stops at the 254th. The \left counter is not recursive, so the stack is not the bound. What was run: - pdflatex (TeX Live 2026), \documentclass{article}\begin{document}$$...$$: 252 ok, 253 ok, 254 and 255 "! TeX capacity exceeded, sorry [grouping levels=255]." inline $...$: 253 ok, 254 capacity exceeded. - Scratch probe (not committed) with both limits at 255: release, 512 KiB thread: tabular/footnote/rotatebox 255 -> stack overflow; colorbox/uline 255 ok; \left 255 ok (also on 256 KiB), 10000 in 120 ms debug, 2 MiB thread: tabular/footnote/colorbox/rotatebox/uline 255 -> stack overflow; \left 255 ok (also on 512 KiB), 10000 in 714 ms - cargo test -p flashtex-compiler --test robustness: test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 36.36s Next step: full compiler and tex-expansion suites and a 5k fuzz smoke run. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: incremental re-expansion equals a full run across stops What changed: - The step limit and the output token limit count from the document start, so a state equivalent at a checkpoint does not decide whether a reused suffix ends the same way. Convergence now also requires the new totals to stay within both limits (or to reach the old run's stop at the same count), and the expander records how its last run ended. - Checkpoints carried past a converged edit get the step offset of the new run; they kept the old run's step count, so a later restart from one stopped at the wrong place (seed 3 edit 54 of the nesting-limit test). - Checkpoints keep the engine's last origin, where a step limit hit on the next step is reported; convergence compares it. - Un-skipped the stop edits in the nesting-limit test; new tests/incremental_stops.rs with minimised regressions and a property test over the fuzz seeds with small limits. What was run: - cargo test --test incremental_stops --test incremental_tests: test result: ok. 6 passed; 0 failed test result: ok. 4 passed; 0 failed - Against the base source: 5 of the 6 incremental_stops tests fail. Next step: full suites, latency bench, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: pin the checkpointed last origin with a test; count stops apart from nesting limits What changed: - step_limit_right_after_a_checkpoint_reports_the_last_origin now edits after the stop with a checkpoint at every safe point, so some cut restarts one step before the limit. - The property test reports stops and nesting-limit excursions separately. What was run: - With the last-origin restore and comparison removed, the test fails: "diagnostics differ -- cut 5". With them: ok. - cargo test --manifest-path crates/tex-expansion/Cargo.toml: test result: ok. 63 passed; 0 failed (expand_tests) test result: ok. 6 passed; 0 failed (incremental_stops) test result: ok. 4 passed; 0 failed (incremental_tests) test result: ok. 2 passed; 0 failed (oracle_tests) - FLASHTEX_INC_STOP_EDITS=60, release: 419 seeds, 25140 edits: 17443 ended at a stop, 6524 went past a nesting limit, 2569 converged; ok. Next step: latency benches, compiler suite, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: incremental edits under new limits equal a full run What changed: - IncrementalExpander::edit_with_limits applies an edit with new step and output-token limits (the compiler's limits grow with the document). A checkpoint is restarted from only if its step count, output count and peak main-memory size fit the new limits; Converge::same_end compares stops against the new limits (a step or output stop is reused when the new total there is the new limit plus one, a main-memory stop only under the same limit, a suffix that did not stop only if its peak fits). Changing the nesting limits re-expands the document. - Engine tracks peak_memory (largest main-memory size checked without stopping) and whether it stopped on main memory; checkpoints keep it. - IncrementalExpander::input_position: where the last run ended, as Engine::input_position gives it for a full run. - tests/incremental_stops.rs: input positions are compared; new tests edits_with_new_limits_cross_every_stop, a_smaller_memory_limit_invalidates_checkpoints_after_a_large_expansion, incremental_matches_full_on_fuzz_seeds_when_limits_change. What was run: - cargo test --release (tex-expansion): all test binaries ok; incremental_stops "test result: ok. 9 passed; 0 failed" - FLASHTEX_INC_STOP_EDITS=30 seeds 1, 2: "419 seeds, 12570 edits with changing limits: 8351 ended at a stop, 972 converged" / "8131 ended at a stop, 960 converged", ok - Each of the 7 new conditions removed in turn: every mutant fails a test. Next step: the compiler's cached expansion uses this and drops its full re-expansion after a stop. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: a stopped cached expansion is not expanded a second time What changed: - expand_project_with_cache no longer re-runs expand_project after the incremental run hits the step limit or TeX capacity (and no longer starts every later revision of a runaway document with a full run). The expander equals a full run across stops (#601), so the cache now types the rest of the entry unexpanded from IncrementalExpander:: input_position, as expand_project does; those tokens are dropped before the next edit is spliced. - The cache expands under expand_project's limits (all project documents' bytes) on every revision via edit_with_limits; it used the entry's size at creation, which only the fallback had hidden. - Debug builds compare every stopped cached run with expand_project. - tests/expansion_incremental.rs: edits_around_a_runaway_loop_match_full_ expansion (typing before/inside/after a loop, same-length replacements that converge with the stopped run, deleting it) and a_runaway_entry_ follows_the_size_of_other_project_documents. What was run: - cargo test --release --test expansion_incremental: "test result: ok. 5 passed; 0 failed"; debug: "ok. 5 passed; 0 failed" (94.64s) - Mutants (no truncation of the recovery, entry-only limits, no recovery) each fail a test. Next step: fixture digests, compiler/tex-expansion suites, before/after timings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler benches: runaway edit scenarios and a digest for one fuzz input What changed: - edit_latency_bench: `HW1 runaway` scenarios (HW1 with a \def\r{x\r}\r loop; typing while it runs away, and deleting/restoring the loop's line) and `--only SUBSTR`. - fuzz_compile `--diagnostics FILE --digest` prints page and diagnostic hashes, to diff two builds on one runaway input. What was run: - edit_latency_bench on 1e4bb85 (#601 head) and this branch: every scenario's reply sha256 is identical (HW1, 500KB, HW1 runaway). - fuzz_compile --diagnostics --digest on fuzz cases 10082 and 18502: same pages and diagnostics digests on both builds. Next step: open the PR with the before/after table. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz --------- Co-authored-by: GoKubar <kabirgoyal@icloud.com>
GoKubar
added a commit
that referenced
this pull request
Sep 15, 2026
… peak back to 1746 MB (stacked on #610) (#619) * compiler: deterministic mutation fuzzer (stable Rust, no new deps) What changed: - tests/fuzz_support/mod.rs: seeds from every .tex under fixtures/ and crates/*/tests; per-case splitmix RNG (seed, index) so every finding replays; mutations: truncate, delete/duplicate brace/bracket/$/\begin/\end tokens, splice lines between files, 10k-deep nesting, 100k-char control sequence names, unbalanced \if/\fi, \def recursion, invalid \char code points, \input of itself. Each case runs compile_full_project (parse + layout) on an 8 MiB thread under catch_unwind with a watchdog; worker processes isolate stack overflows and hangs; ddmin minimiser. - examples/fuzz_compile.rs: supervisor, --replay, --check, --minimise, --diagnostics. - tests/fuzz_compile.rs: the same run as an #[ignore] test (debug build). What was run: - fuzz_compile --cases 300 --jobs 12: "fuzz: 300 cases, 4 unique findings" - FLASHTEX_FUZZ_CASES=40 cargo test --test fuzz_compile -- --ignored: "the fuzzer found 3 unique failures" (expected before the fixes) Next step: 50k-case run, minimise and fix each finding. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: lists nested past 255 levels are "Too deeply nested", not a u8 overflow What changed: - The per-kind and total list depths were `count() as u8 + 1` in push_list_frame and open_list: 255 enclosing lists panicked with "attempt to add with overflow" (fuzz finding). Depths now saturate (next_list_depths), and a \begin past LaTeX's \@toodeep limits (six \list levels; four itemize/enumerate) reports "LaTeX Error: Too deeply nested." while the list is still typeset. - enumitem start=<i64::MIN> no longer overflows `n - 1`. - Regression test in tests/robustness.rs. What was run: - cargo test --test robustness lists_nested (before the fix): "panicked at src/parser.rs:6972:26: attempt to add with overflow" - after: "test result: ok. 1 passed; 0 failed" Next step: remaining fuzz findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: an unclosed math span never ends before its opener What changed: - finish_math took an unclosed formula's end from the last content token. When the content was re-read from earlier input (`\setlength{\begin{}$`) that token lay before the `$`, so Span::in_document tripped "span start must not exceed end" (debug) or produced an inverted span (release). The end now comes only from the opener's document and never precedes the opener's end. - Regression test an_unclosed_math_span_never_inverts. What was run: - before: "panicked at src/lib.rs:60:9: span start must not exceed end" - after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed" Next step: the cross-document alignment span. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: an alignment that \input's another document keeps its spans in one document What changed: - multirow_environment merged every cell token's span into the row span; with `\begin{align}a\include{sub}` the cells hold tokens of sub.tex and main.tex, tripping "cannot merge spans from different documents". Row spans now merge only tokens of the environment's own document, and an `\end{align}` found in another document no longer sets the display's end offset (it could invert the span). - Regression test an_alignment_that_inputs_another_document_keeps_its_spans_in_one_document. What was run: - before: "panicked at src/lib.rs:74:9: assertion `left == right` failed: cannot merge spans from different documents" - after: cargo test --test robustness: "test result: ok. 9 passed; 0 failed" Next step: remaining fuzz findings (tex-expansion overflow, stack overflow, hangs). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: TeX's integer and dimension ranges instead of i64 overflow What changed: - Fuzz finding: `\advance` added to a count holding i64::MAX (an over-long digit string parsed to i64::MAX) and panicked with "attempt to add with overflow" (expand.rs do_arith). - Constants follow tex.web §445: past 2^31-1 is "Number too big." and becomes 2147483647 (decimal, octal, hex; also the integer part of a dimen/glue, which silently became 0 before). - `<dimen>` past max_dimen is "Dimension too large." and becomes ±max_dimen (§448). - \advance and \multiply check the result against infinity / max_dimen; on overflow they report "Arithmetic overflow." and assign nothing (§1236-1240). - \numexpr/\dimexpr: saturating steps, and an out-of-range result is "Arithmetic overflow." with value 0 (e-TeX); rounded_div computes in i128. - `fil` followed by more than two further `l`s overflowed the u8 order; now "Illegal unit of measure (replaced by filll)." (§454). - Regression test numeric_ranges_follow_tex_instead_of_overflowing. What was run: - before: left "9223372036854775807" (and the add-overflow panic) - after: cargo test (tex-expansion): "test result: ok. 56 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: stack overflow and hang findings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: nested sub-parses stop at TeX's 255 grouping levels, not the stack What changed: - Fuzz finding: a table cell, footnote, \colorbox/\rotatebox/\uline argument or \include re-enters parse_stream on its own token stream with no depth bound. 3000 nested tabulars overflowed an 8 MiB release stack ("thread has overflowed its stack"), and 10k took minutes (every level copies its cell tokens). - parse_stream now counts its nesting; past STREAM_DEPTH_LIMIT = 255 it reports "TeX capacity exceeded, sorry [grouping levels=255]." once per compile and skips the content nested past the limit. - fuzz_support: FLASHTEX_FUZZ_STACK_KB overrides the case stack size. - Regression test nested_sub_parses_hit_tex_grouping_capacity_instead_of_the_stack (debug frames are ~50 KiB per tabular level, so it runs on a 64 MiB thread). What was run: - before: "thread '<unknown>' has overflowed its stack / fatal runtime error" - after: "test result: ok. 1 passed; 0 failed" (5.86s, debug) - release --check at depth 255, 2 MiB stack: "signature: ok" (tabular, footnote) Next step: hangs (\loop runaway, unknown-command suggestion cost). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: a macro call cut off by the end of file is aborted (tex.web §339) What changed: - Fuzz finding (hang): `\loop{\n}` with no \repeat. The file ended while \loop's argument was scanned; the expander reported the runaway but still expanded the body with the partial argument, so \iterate ran until the 2M expansion-step limit, emitting "Extra \fi." (and, via the compiler, one unknown-command diagnostic) per iteration: >2 minutes to compile. - TeX sets long_state := outer_call there, so the inserted \par aborts the call. report_file_ended_and_recover now sets the silent runaway flag for Matching, and call_macro aborts. - Regression test a_macro_call_cut_off_by_the_end_of_file_is_aborted. What was run: - before: diagnostics ["Runaway argument?...", "Extra \fi.", "Extra \fi.", ...] - after: cargo test (tex-expansion): "ok. 57 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: the unknown-command suggestion cost behind the remaining hangs. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: unknown-command suggestions are memoised, so a runaway loop cannot hang What changed: - Fuzz finding (hang): `\def\a{\n\a}\a` emits ~666k `\n` before the 2M expansion-step limit. Each got an unknown-command diagnostic whose did-you-mean scanned the whole vocabulary several times (command_error -> closest_commands, command_help -> suggest_command, is_known_command), allocating per candidate: 269 s in release. - closest_commands is memoised per thread (bounded at 4096 names); is_known_command is a HashSet lookup; candidates whose length differs by more than the edit limit are skipped before edit_distance copies the name (a 100k-character control sequence: 860 ms -> 6 ms). - Regression test a_runaway_loop_of_unknown_commands_is_diagnosed_in_bounded_time. What was run: - release, before: "269114 ms, 1 pages, 666816 diagnostics" - release, after: "1989 ms, 1 pages, 666816 diagnostics" - debug test after: "test result: ok. 1 passed; 0 failed ... finished in 10.88s" Next step: re-run the 50k fuzz on the fixed build. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: a macro expansion past the token budget is "TeX capacity exceeded" What changed: - Fuzz finding (hang, 9 of 26 hangs in a 2000-case run): `\def\a#1{\a{#1#1}}\a x` doubles its argument on every call, so the token lists exhaust memory and time after ~30 calls while the 2M expansion-step limit is nowhere near. - call_macro sizes the substitution before building it; past the output-token budget (Limits::max_output_tokens) it reports "TeX capacity exceeded, sorry [main memory size=5000000]." and stops expansion, as TeX does when main memory runs out. - Regression test an_argument_that_doubles_every_call_exceeds_capacity. What was run: - before (release fuzz build, --check, 40 s watchdog): cases 1204 1196 1599 1849 567 894 393 941 1280 "signature: hang" - after: all nine "signature: ok" in <= 1 s - cargo test (tex-expansion): "ok. 58 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: the slow deep-nesting cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: argument and display spans never invert on replayed tokens What changed: - Reported from another lane: parsing crates/page-builder/oracle/fixtures/vspace-03.tex panicked with "span start must not exceed end" in required_group_bounded. Minimised: `\AtBeginDocument{\setlength\}}` + `\begin{document`. The replayed argument's closing offset lies before its `{`. - One helper, P::span_through(open, end), now builds every "opener through end offset" span (required_group_bounded, display math, alignments, tabular, the delimited-argument reader): the end never precedes the opener's end or runs past the opener's document. - Regression test an_argument_replayed_from_at_begin_document_never_inverts_its_span (minimised input plus the whole fixture). What was run: - before: "panicked at src/lib.rs:60:9: span start must not exceed end" - after: "test result: ok. 1 passed; 0 failed" Next step: \loop oracle fixtures parse time; slow deep-nesting cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler fuzz: seed from crates/*/oracle too; keep a sample of hang/crash inputs What changed: - Seeds now include crates/*/oracle/**/*.tex (the page-builder oracle fixture vspace-03.tex panicked the parser and was not a seed). This changes which input each case index generates. - Up to 50 inputs per hang/crash signature are saved for triage (those signatures do not distinguish causes). What was run: - fuzz_compile --cases 2000: "fuzz: 2000 cases, 1 unique findings" (26 hangs, triaged) Next step: full 50k re-run on the fixed build. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: sub-parse depth limit 32 (fits 512 KiB release and 2 MiB debug threads) What changed: - Measured after the previous commit: 255 nested tabulars still overflow a 512 KiB thread in release (128 already did; ~7 KiB a level) and a 2 MiB debug thread fails at 40 levels (~50 KiB a level), so the guard did not fire first on every profile. STREAM_DEPTH_LIMIT is now 32, like MAX_MATH_DEPTH; the error reads "TeX capacity exceeded, sorry [grouping levels=32]." - \include nesting keeps its own INCLUDE_DEPTH_LIMIT (64) and no longer counts toward the sub-parse depth. - The regression test runs on the default test thread again. What was run: - release fuzz build, FLASHTEX_FUZZ_STACK_KB=512, 3000 nested tabular / footnote: "signature: ok" (both) - cargo test --test robustness nested_sub_parses: "test result: ok. 1 passed" (0.28s) Next step: \left...\right nesting is quadratic in layout. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: \left...\right nesting past 32 levels is a capacity error, not quadratic layout What changed: - Fuzz finding (slow compile, 15.7 s release for one input): sizing a \left/\right pair lays out all the atoms it encloses (left_right_stretch_scales), so 10k nested pairs cost depth x length. - The math parser counts open \left's; past MAX_LEFT_RIGHT_DEPTH = 32 it reports "TeX capacity exceeded, sorry [grouping levels=32]." once and drops the extra \left's and their \right's. Formulas within the limit are unchanged (left_right_nesting goldens pass). - Regression test deeply_nested_left_right_pairs_hit_a_capacity_limit_in_bounded_time. What was run: - before: fuzz case 529 "15664 ms, 2 pages, 17120 diagnostics" (release) - after: debug test with 10k nested pairs "ok. 1 passed ... finished in 0.13s" - cargo test --test left_right_nesting: "test result: ok. 3 passed; 0 failed" Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: the recursion-limit unit test uses a name the kernel does not define What changed: - self_referential_macro_hits_explicit_recursion_limit used `\newcommand{\loop}{\loop} \loop`. The prelude already defines \loop, so \newcommand keeps `\loop#1\repeat`, whose argument runs to the end of the file. It only reached the step limit through the bug fixed in "tex-expansion: a macro call cut off by the end of file is aborted"; TeX reports the runaway argument instead. The test now uses \recurse. What was run: - cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast: 58 suites, 676 passed, 0 failed, 6 ignored Next step: 50k fuzz re-run, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler fuzz: rustfmt the harness and robustness tests What changed: - rustfmt on the new fuzz files and tests/robustness.rs (which was rustfmt-clean on main). No behaviour change. What was run: - cargo test --test robustness --test fuzz_compile: "test result: ok. 13 passed" and "ok. 0 passed; 0 failed; 1 ignored" Next step: PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: input stack and pending-token capacity, as TeX's "capacity exceeded" What changed: - Fuzz finding (36 GB resident, killed): `\def\a{\csname a\endcsna` + 10k `[` ... `]` + `me}\a`. `\csname a` re-enters \a before the rest of its 20k-token body is read, so every call adds an input level holding that body; memory ran out long before the 2M expansion-step limit. - Pushing a token list now checks TeX's limits: 10000 input levels ("TeX capacity exceeded, sorry [input stack size=10000].", TeX Live's stack_size) and, every 64 levels, the tokens still pending across all levels against the output-token budget ("TeX capacity exceeded, sorry [main memory size=5000000]."). Either stops expansion. - Regression test a_non_tail_self_call_exceeds_capacity_instead_of_memory. What was run: - before: /usr/bin/time -l fuzz_compile --check case-14361: "36046176256 maximum resident set size", 80 s, killed (exit 137) in a second run - after: "30 ms, 1 pages, 133 diagnostics", "172441600 maximum resident set size" - cargo test (tex-expansion): "ok. 59 passed; 0 failed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: remaining slow cases; full compiler suite; PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: list levels past \@toodeep are counted, not copied into every block What changed: - Fuzz finding (25.7 GB resident): 30k nested itemize. Every block clones the enclosing list frames, so memory was blocks x depth. - push_list_frame now stores no frame past LaTeX's limit (the one that reports "LaTeX Error: Too deeply nested."); it counts dropped levels, and a list \end pops a dropped level first. Blocks hold at most six frames; the dropped levels typeset at the deepest stored level. - lists_nested_past_255_levels_are_too_deeply_nested_not_an_overflow also checks that no block stores more than six frames. What was run: - before: fuzz case 12106 "4117 ms, 884 pages", "25687048192 maximum resident set size"; test: "a block stored 7 list frames" - after: "622 ms, 884 pages", "138084352 maximum resident set size"; test "ok. 1 passed" - cargo test --test list_structure --test setlist_spacing --test acceptance --test pinned_fixtures --lib: all "test result: ok" Next step: remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: an \if's line is computed only when its error is reported What changed: - Fuzz finding (22 s): a mutated microtype oracle fixture whose \loop never ends. do_conditional (and a taken branch's \else) called line_of_span on every \if, counting newlines from the start of the file, for the "Incomplete \if...; all text was ignored after line N." message that only an unterminated conditional prints: quadratic in a runaway loop. - ScannerStatus::Skipping now carries the \if's span; the line is counted when that error is reported. The unused line count on a mismatched \end is gone. - Regression test conditionals_in_a_runaway_loop_do_not_rescan_the_source (also pins the "after line 3." wording). What was run: - before: test panicked "244.136504125s" (debug, 30 s budget) - after: "test result: ok. 1 passed ... finished in 0.53s"; tex-expansion cargo test: "ok. 60 passed" (expand_tests), "ok. 3 passed" (incremental), "ok. 2 passed" (oracle) Next step: re-time the remaining slow fuzz cases. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: a TeX capacity stop is recovered like the expansion step limit What changed: - The new tex-expansion capacity stops ("TeX capacity exceeded, sorry [input stack size=10000]." / "[main memory size=5000000].") end expansion at a point that depends on where the run started. The expansion cache only knew the step limit, so an incremental run could stop elsewhere than a full one: expansion_incremental's random-edit test diverged at step 198 ("tokens differ at 60356 (cached 60356 vs full 60386 tokens)"). The rest of the document was also dropped instead of typeset unexpanded, and the recovery note said "continued expanding". - step_limit_hit (cache fallback, unexpanded resume) and recovery_for now treat both stops alike. - Regression test a_capacity_stop_is_recovered_like_the_step_limit. What was run: - before: expansion_incremental FAILED as above; robustness test: left Some("continued expanding after the problem") - after: cargo test --test expansion_incremental: "ok. 2 passed; 0 failed"; cargo test --test robustness a_capacity_stop: "ok. 1 passed" Next step: full compiler suite, final fuzz run, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler, tex-expansion: bound diagnostic floods from runaway loops What changed: - tex-expansion: an identical diagnostic (severity, message, span) is recorded once between safe points; the set is emptied when next_content_token returns at a safe point, which is where incremental checkpoints are taken, so restored engines decide exactly as a full run. - tex-expansion: "group nesting limit exceeded" / "conditional nesting limit exceeded" are reported once per excursion (flags in checkpointed State). - tex-expansion: \end compares @Currenvir part by part without rebuilding the name; environment names longer than 100 characters are shortened in messages only. - compiler: diagnostics::limit_repeats drops identical repeats and keeps 1000 distinct diagnostics per code, then one "further N similar diagnostics suppressed" summary per code; fatal stops are always kept. Applied to the parser's list and to the layout's list, identically in clean and incremental compiles. - compiler: unsupported / unsupported_preamble skip building a repeat of a command already reported at the same span. - fuzz harness: per-case diagnostic count and time, max diagnostics and slowest case in the report; example --diagnostics prints the most frequent messages; --digest hashes pages and diagnostics of every seed. - tests: diagnostic_floods.rs (child process per case, peak heap via a counting allocator, watchdog), engine excursion/dedupe/long-name tests, incremental equivalence with repeats and nesting limits, warm-vs-fresh JSON identity with bounded diagnostics. What was run: - cargo test --manifest-path crates/tex-expansion/Cargo.toml: test result: ok. 63 passed; 0 failed (expand_tests), test result: ok. 4 passed; 0 failed (incremental_tests), test result: ok. 2 passed; 0 failed (oracle_tests) - cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast: 59 suites, 684 passed, 0 failed, 6 ignored - fuzz_compile --cases 50000 --jobs 6 --seed 0xF1A57E40 (overflow checks, debug assertions): 0 panics, 0 overflows, 14 hangs (base 18); max diagnostics in one case 3109 (base 2974694) - fuzz_compile --digest over 419 seeds: pages identical for all 419; diagnostics differ in 49 (48 \loop fixtures ~14750 -> ~134, and 20-Vmatrix 3 -> 2, a duplicate glyph note) Next step: merge the updated compiler-fuzz branch and open the PR stacked on #577. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: \advance wraps in 32-bit arithmetic like pdfTeX, with no diagnostic What changed: - `\advance` on count, dimen and skip registers (value, stretch and shrink) adds with explicit i32 wrapping (tex.web §1238-1239 has no range check) instead of reporting "Arithmetic overflow." and leaving the register. `\multiply`, `\divide`, `\numexpr` and `\dimexpr` keep their bounds checks. - numeric_ranges_follow_tex_instead_of_overflowing pins pdfTeX's wrapped values with no diagnostic. What was run: - pdftex 3.141592653-2.6-1.40.29 (TeX Live 2026), plain and -etex: \count0=2147483647 \advance\count0 by 1 \showthe\count0 -> -2147483648 \count0=-2147483647 \advance\count0 by -2 -> 2147483647 \dimen0=\maxdimen \advance\dimen0 by 1sp \showthe\dimen0 -> 16384.0pt \dimen0=16383pt \advance\dimen0 by 16383pt -> 32766.0pt 3x\maxdimen -> -16384.00005pt; 4x\maxdimen+1sp -> -0.00005pt skip \maxdimen plus \maxdimen minus 1pt, advanced by the same minus 2pt -> 32767.99997pt plus 32767.99997pt minus 3.0pt - cargo test -p flashtex-tex-expansion --test expand_tests: test result: ok. 60 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out Next step: reword or raise the grouping-level capacity limit (#577 review). Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: the sub-parse depth limit is FlashTeX's; \left nesting stops where pdflatex does What changed: - STREAM_DEPTH_LIMIT stays 32 (255 nested tabulars, footnotes or rotateboxes overflow a 512 KiB release and a 2 MiB debug thread), and its error now says "FlashTeX nesting limit (32) exceeded" instead of claiming TeX's "grouping levels" capacity. - MAX_LEFT_RIGHT_DEPTH rises from 32 to 253 (TEX_GROUPING_LEVELS - 2), with "TeX capacity exceeded, sorry [grouping levels=255].": pdflatex accepts 253 nested \left in a document's $$ or $ math and stops at the 254th. The \left counter is not recursive, so the stack is not the bound. What was run: - pdflatex (TeX Live 2026), \documentclass{article}\begin{document}$$...$$: 252 ok, 253 ok, 254 and 255 "! TeX capacity exceeded, sorry [grouping levels=255]." inline $...$: 253 ok, 254 capacity exceeded. - Scratch probe (not committed) with both limits at 255: release, 512 KiB thread: tabular/footnote/rotatebox 255 -> stack overflow; colorbox/uline 255 ok; \left 255 ok (also on 256 KiB), 10000 in 120 ms debug, 2 MiB thread: tabular/footnote/colorbox/rotatebox/uline 255 -> stack overflow; \left 255 ok (also on 512 KiB), 10000 in 714 ms - cargo test -p flashtex-compiler --test robustness: test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 36.36s Next step: full compiler and tex-expansion suites and a 5k fuzz smoke run. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: incremental re-expansion equals a full run across stops What changed: - The step limit and the output token limit count from the document start, so a state equivalent at a checkpoint does not decide whether a reused suffix ends the same way. Convergence now also requires the new totals to stay within both limits (or to reach the old run's stop at the same count), and the expander records how its last run ended. - Checkpoints carried past a converged edit get the step offset of the new run; they kept the old run's step count, so a later restart from one stopped at the wrong place (seed 3 edit 54 of the nesting-limit test). - Checkpoints keep the engine's last origin, where a step limit hit on the next step is reported; convergence compares it. - Un-skipped the stop edits in the nesting-limit test; new tests/incremental_stops.rs with minimised regressions and a property test over the fuzz seeds with small limits. What was run: - cargo test --test incremental_stops --test incremental_tests: test result: ok. 6 passed; 0 failed test result: ok. 4 passed; 0 failed - Against the base source: 5 of the 6 incremental_stops tests fail. Next step: full suites, latency bench, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: pin the checkpointed last origin with a test; count stops apart from nesting limits What changed: - step_limit_right_after_a_checkpoint_reports_the_last_origin now edits after the stop with a checkpoint at every safe point, so some cut restarts one step before the limit. - The property test reports stops and nesting-limit excursions separately. What was run: - With the last-origin restore and comparison removed, the test fails: "diagnostics differ -- cut 5". With them: ok. - cargo test --manifest-path crates/tex-expansion/Cargo.toml: test result: ok. 63 passed; 0 failed (expand_tests) test result: ok. 6 passed; 0 failed (incremental_stops) test result: ok. 4 passed; 0 failed (incremental_tests) test result: ok. 2 passed; 0 failed (oracle_tests) - FLASHTEX_INC_STOP_EDITS=60, release: 419 seeds, 25140 edits: 17443 ended at a stop, 6524 went past a nesting limit, 2569 converged; ok. Next step: latency benches, compiler suite, PR. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion: incremental edits under new limits equal a full run What changed: - IncrementalExpander::edit_with_limits applies an edit with new step and output-token limits (the compiler's limits grow with the document). A checkpoint is restarted from only if its step count, output count and peak main-memory size fit the new limits; Converge::same_end compares stops against the new limits (a step or output stop is reused when the new total there is the new limit plus one, a main-memory stop only under the same limit, a suffix that did not stop only if its peak fits). Changing the nesting limits re-expands the document. - Engine tracks peak_memory (largest main-memory size checked without stopping) and whether it stopped on main memory; checkpoints keep it. - IncrementalExpander::input_position: where the last run ended, as Engine::input_position gives it for a full run. - tests/incremental_stops.rs: input positions are compared; new tests edits_with_new_limits_cross_every_stop, a_smaller_memory_limit_invalidates_checkpoints_after_a_large_expansion, incremental_matches_full_on_fuzz_seeds_when_limits_change. What was run: - cargo test --release (tex-expansion): all test binaries ok; incremental_stops "test result: ok. 9 passed; 0 failed" - FLASHTEX_INC_STOP_EDITS=30 seeds 1, 2: "419 seeds, 12570 edits with changing limits: 8351 ended at a stop, 972 converged" / "8131 ended at a stop, 960 converged", ok - Each of the 7 new conditions removed in turn: every mutant fails a test. Next step: the compiler's cached expansion uses this and drops its full re-expansion after a stop. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: a stopped cached expansion is not expanded a second time What changed: - expand_project_with_cache no longer re-runs expand_project after the incremental run hits the step limit or TeX capacity (and no longer starts every later revision of a runaway document with a full run). The expander equals a full run across stops (#601), so the cache now types the rest of the entry unexpanded from IncrementalExpander:: input_position, as expand_project does; those tokens are dropped before the next edit is spliced. - The cache expands under expand_project's limits (all project documents' bytes) on every revision via edit_with_limits; it used the entry's size at creation, which only the fallback had hidden. - Debug builds compare every stopped cached run with expand_project. - tests/expansion_incremental.rs: edits_around_a_runaway_loop_match_full_ expansion (typing before/inside/after a loop, same-length replacements that converge with the stopped run, deleting it) and a_runaway_entry_ follows_the_size_of_other_project_documents. What was run: - cargo test --release --test expansion_incremental: "test result: ok. 5 passed; 0 failed"; debug: "ok. 5 passed; 0 failed" (94.64s) - Mutants (no truncation of the recovery, entry-only limits, no recovery) each fail a test. Next step: fixture digests, compiler/tex-expansion suites, before/after timings. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler benches: runaway edit scenarios and a digest for one fuzz input What changed: - edit_latency_bench: `HW1 runaway` scenarios (HW1 with a \def\r{x\r}\r loop; typing while it runs away, and deleting/restoring the loop's line) and `--only SUBSTR`. - fuzz_compile `--diagnostics FILE --digest` prints page and diagnostic hashes, to diff two builds on one runaway input. What was run: - edit_latency_bench on 1e4bb85 (#601 head) and this branch: every scenario's reply sha256 is identical (HW1, 500KB, HW1 runaway). - fuzz_compile --diagnostics --digest on fuzz cases 10082 and 18502: same pages and diagnostics digests on both builds. Next step: open the PR with the before/after table. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * compiler: the full expansion stops on the output token limit; one allocation for the cached stream What changed: - expand_project counts engine tokens and stops once the count goes past max_output_tokens, with the incremental expander's "output token limit exceeded" diagnostic, after converting the token that went past it. Before, only the incremental path had this limit, so a small (< 4 KB, full path) document with an output-heavy loop ran on to the step limit and typeset the rest unexpanded, while a larger one stopped earlier. - The debug-build parity check in expand_project_with_cache also covers output-limit stops. - build_cache reserves the converted stream once (engine tokens + bytes of the unexpanded rest) and finish trims it to at most 1/8 spare. Growing it by doubling left a freed chain of blocks resident for the rest of the compile: fuzz case 18502's peak footprint 2146 MB -> 1746 MB (#601: 1744 MB). - Tests: an_output_heavy_loop_stops_both_paths_at_the_output_token_limit and output_heavy_runaway_loops_match_full_expansion_under_random_edits. What was run: - cargo test --release --test expansion_incremental: test result: ok. 7 passed; 0 failed - Mutation (no output limit on the full path): both new tests fail (tokens differ at 763823, cached 763823 vs full 801109 tokens). - fuzz_compile --digest, 419 seeds: shasum 36b592ea57085d703ffea84d4b6979c628dbf236, identical to #610's head. Next step: suites, edit_latency_bench, PR stacked on #610. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion, compiler: the output token limit's diagnostic says the rest was not typeset What changed: - The output token limit's diagnostic is now, in TeX's capacity wording, "TeX capacity exceeded, sorry [output token limit=N]; expansion stopped here and the rest of the document was not typeset." (was "output token limit exceeded"). One definition, tex::output_limit_message, used by IncrementalExpander, Engine::run and expand_project; tex::is_output_limit matches it. - Its compiler recovery note is "stopped expanding; the rest of the document was not typeset" (was "...typeset without macro expansion", which neither path does after this stop). - The message now starts with "TeX capacity exceeded, sorry [", so the unexpanded recovery (step_limit_hit) excludes it explicitly; is_stop_limit still matches it, so the diagnostic flood cap always keeps it. - The step limit's text is unchanged: it does typeset the rest unexpanded. - Tests assert the exact message and recovery, and that nothing after the loop is typeset. What was run: - tex-expansion cargo test --release: 0 + 63 + 9 + 4 + 2 passed, 0 failed - compiler cargo test --release: 59 test binaries, 688 passed, 0 failed, 6 ignored - expansion_incremental (debug): test result: ok. 7 passed; 0 failed (98.57s) - Mutation (output-limit stop resumes unexpanded): an_output_heavy_loop_stops_both_paths_at_the_output_token_limit fails. - fuzz_compile --digest, 419 seeds: 36b592ea57085d703ffea84d4b6979c628dbf236, unchanged. - bodyloop.tex: pages 163c9d2a21b1822b unchanged; diagnostics d82176b9e51d82eb -> 856cb57fcc2fede2 (message text only). Next step: reply on #619. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz * tex-expansion, compiler: nesting-limit recovery notes say expansion continued What changed: - "group nesting limit exceeded" and "conditional nesting limit exceeded" got the note "stopped expanding; the rest of the document was typeset without macro expansion" (recovery_for matched contains("limit exceeded")). The engine does not stop there: maybe_handle_brace drops the `{` without opening a group, and do_conditional returns before pushing the conditional, so its test is read as text; expansion continues in both cases. - New notes: "the extra group was ignored and expansion continued" and "the extra conditional was ignored without evaluating its test, and expansion continued". The step-limit branch now matches only is_stop_limit, whose note (rest typeset unexpanded) is true. - Messages are unchanged (not TeX capacity wording: that prefix means a stop to is_stop_limit and the flood cap). - Tests: expand_tests past_a_nesting_limit_the_extra_group_or_conditional_is_ignored_and_expansion_continues pins the engine behaviour (tokens and follow-on diagnostics); expansion_incremental expansion_limit_notes_describe_what_happens pins the three notes. What was run: - tex-expansion cargo test --release: 0 + 64 + 9 + 4 + 2 passed, 0 failed - compiler cargo test --release: 59 test binaries, 689 passed, 0 failed, 6 ignored - fuzz_compile --digest, 419 seeds: 36b592ea57085d703ffea84d4b6979c628dbf236, unchanged. - Diagnostics hashes change (pages unchanged) for 10082 9232f3ba710cfc05 -> f0c08aabd804800a, conds.tex f0075201d8ee58ff -> 1d31e6378d071330, groups.tex 78f34e551c859d61 -> cd117cd5038b2f6c. Next step: reply on #619. Implementation-Agent: claude-opus-5 subagent of daniel-parent Commit-Executor: daniel-parent subagent Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz --------- Co-authored-by: GoKubar <kabirgoyal@icloud.com>
GoKubar
added a commit
that referenced
this pull request
Sep 15, 2026
* compiler: implement \thepage via the \pageref late-binding path
\thepage parses to a new Inline::ThePage resolved at layout time to the
physical page being set, formatted in the \pagenumbering style in force
there (previously: unknown_command). \pagenumbering now emits a
zero-width Inline::PageNumbering marker that resets the displayed page
counter to 1 and switches the style; labels capture the styled page text
so \pageref (and cleveref page refs) match. Unknown styles fall back to
arabic, keeping the command accepted as before.
Implementation-Agent: muse-spark-1.3-contributor
Commit-Executor: muse-spark-1.3-contributor
* compiler: support \captionof{figure/table}[short]{text} outside floats
caption.sty's standalone caption form reuses \caption's exact machinery
(counter step, Figure/Table N: prefix, FigureCaption block) with the float
type given explicitly instead of read from the enclosing environment.
The [short] list-of-figures argument is consumed and ignored; unknown
types diagnose and fall back to paragraph text like \caption outside a
figure. Inventory entry plus regenerated supported-LaTeX artifacts.
Implementation-Agent: muse-spark-1.3-contributor
* compiler: parse \marginpar and set the note in the right margin (#505)
\marginpar[<left>]{<right>} now parses to a new Inline::Marginpar
carrying the one-sided {<right>} note (a present [<left>] is consumed
with a warning); the unsupported_feature diagnostic is gone in the real
compiler, incremental span-shifting/block signatures and the Core 14
layout (skips the note, no mark) updated. Stated simplification: always
the right margin (no two-sided alternation, \reversemarginpar), no
collision avoidance between close notes.
Render-pipeline placement lands separately (needs a vendor/compiler
re-pin past this commit to compile the new Inline variant) — see the
paired draft PR.
Implementation-Agent: muse-spark-1.3-contributor
Commit-Executor: daniel-muse-lead-2 (Claude Sonnet Muse supervisor; split
compiler-only files from muse/marginpar 3473c632/9c819976 for the
compiler-PR-first/pipeline-draft-second pattern)
Co-authored-by: d-q222 <279808976+d-q222@users.noreply.github.com>
* compiler: add basic biblatex support
What changed:
- add project-relative biblatex resource loading, numeric citations, notes, sorting, and bibliography rendering
- route biblatex commands through a dedicated compiler module
What was run:
- CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-biblatex cargo test --manifest-path crates/compiler/Cargo.toml --test biblatex -- --nocapture: 7 passed
Next step:
- commit focused coverage and regenerated support inventories
Lane-Owner: daniel-parent (mac-m5pro-dq222)
Implementation-Agent: codex gpt-5.6-luna (max)
Commit-Executor: codex (this session); trailers added by daniel-parent
Lane-Owner: daniel-parent (mac-m5pro-dq222)
* compiler: cover biblatex support
What changed:
- add coverage for citations, notes, textcite, nocite, sorting, undefined keys, and headings
- regenerate compiler and macOS supported-LaTeX inventories and user docs
What was run:
- CARGO_TARGET_DIR=/Users/dqi26/flashtex/target-biblatex cargo test --manifest-path crates/compiler/Cargo.toml --test biblatex -- --nocapture: test result: ok. 7 passed; 0 failed
- sh crates/compiler/scripts/render_supported_latex.sh: completed
Next step:
- run the final full compiler test suite
Lane-Owner: daniel-parent (mac-m5pro-dq222)
Implementation-Agent: codex gpt-5.6-luna (max)
Commit-Executor: codex (this session); trailers added by daniel-parent
Lane-Owner: daniel-parent (mac-m5pro-dq222)
* compiler: route eqnarray/eqnarray* through multirow displays (#520)
eqnarray/eqnarray* now parse via multirow_environment (rows on \\, 3
cells on &), numbered against the shared equation counter (* unnumbers
all, \nonumber/\notag unnumbers one row); N rows consume N+1 numbers
per ltmath.dtx. Both envs moved from the unimplemented to the
implemented inventory.
Render-pipeline layout (RowsEnv::EqnArray, right/centred/left columns,
kernel arraycolsep spacing) lands separately — see the paired draft PR;
it has no compile-time dependency on this commit (RowsEnv reads the
environment name from source text), but its end-to-end geometry test
stays #[ignore]d until a vendor/compiler re-pin lets it see real
eqnarray output instead of the vendored plain-text fallback.
Implementation-Agent: muse-spark-1.3-contributor
Commit-Executor: daniel-muse-lead-2 (Claude Sonnet Muse supervisor; split
compiler-only files from muse/eqnarray for the compiler-PR-first/
pipeline-draft-second pattern)
Co-authored-by: d-q222 <279808976+d-q222@users.noreply.github.com>
* compiler: keep biblatex's parser arms out of the recursive command frame
What changed:
- `\addbibresource` and `\printbibliography` move out of `P::command` into
`#[inline(never)]` helpers (`add_bib_resource`, `print_bibliography`);
`biblatex_cite` is `#[inline(never)]` too.
- The biblatex `\citeauthor`/`\citeyear` dispatch moves from the `command`
arm to the top of `natbib_cite`.
- Why: `P::command` is on the stack once per nested sub-parse. In debug
builds every local of every arm gets its own slot, so #558's inline arms
grew each nesting level by 1440 bytes. Combined with #601, 32 nested
tabulars (#577's STREAM_DEPTH_LIMIT, kept at 32) overflowed the 2 MiB
debug test thread in
robustness::nested_sub_parses_hit_tex_grouping_capacity_instead_of_the_stack
(found by the integration preview, #609).
What was run:
- Stack per nesting level (debug), measured with a scratch probe that prints
the stack address in `parse_stream` at each depth (not committed):
| construct | #601 | #601+#558 before | #601+#558 after |
| tabular | 57232 | 58672 | 57360 |
| footnote | 31424 | 32864 | 31552 |
| colorbox | 30928 | 32368 | 31056 |
| rotatebox | 32048 | 33488 | 32176 |
| uline | 30336 | 31776 | 30464 |
- #601 + #558 (merge), debug, CARGO_BUILD_JOBS=4:
cargo test --test robustness --test biblatex
biblatex: test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
robustness: test result: ok. 14 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 41.06s
- This branch, debug:
biblatex: test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
robustness: test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 40.81s
natbib: test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
--lib cite: test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 331 filtered out; finished in 0.00s
Next step: merge into the integration preview (#609) and rerun there.
Implementation-Agent: claude-opus-5 subagent of daniel-parent
Commit-Executor: daniel-parent subagent
Claude-Session: https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz
* integration: regenerate supported-latex artifacts on the combined head
crates/compiler/scripts/render_supported_latex.sh over the five merged
PRs: 333 -> 341 text commands, 62 -> 64 environments (eqnarray,
eqnarray*), 23 -> 24 packages (biblatex); Mac bundled copy synced.
Implementation-Agent: fable-integration-cmp
Commit-Executor: fable-integration-cmp (direct; Cursor usage limit)
* integration: regenerate bridge and paragraph-layout lockfiles for flashtex-bibliography
PR #558 added flashtex-bibliography to flashtex-compiler's dependencies
but only updated crates/compiler/Cargo.lock. bridge and paragraph-layout
path-depend on the real compiler, so their lockfiles gain the same entry;
flashtex-cli and render-pipeline build against the vendored compiler
snapshot and are unchanged. Verified CI-style with
cargo build --release --locked for all five crates.
Implementation-Agent: fable-integration-cmp
Commit-Executor: fable-integration-cmp (direct; Cursor usage limit)
---------
Co-authored-by: muse-spark-1.3-contributor <muse-contributor@flashtex.invalid>
Co-authored-by: d-q222 <279808976+d-q222@users.noreply.github.com>
Contributor
|
Landed via #610 ( GitHub could not auto-close this because #610 was squash-merged, so this branch's tip is not an ancestor of main even though its content is. Verified present on main before closing:
Worth recording why this was closed rather than re-merged: an integration lane attempted to include these branches and found that doing so would have regressed main — their heads still carry the older fuzz tooling (no — orchestrator-astra (Commander) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A deterministic mutation fuzzer for the compiler that runs on stable Rust with no new dependencies, plus fixes for every panic, stack overflow, hang and memory blow-up it found. It also fixes the two robustness reports from other lanes:
vspace-03.texand the slow\looporacle fixtures.Harness (
crates/compiler/tests/fuzz_support/mod.rs,examples/fuzz_compile.rs,tests/fuzz_compile.rs#[ignore]):.texunderfixtures/,crates/*/tests/andcrates/*/oracle/(419 files).--replay N.{ } [ ] $ \begin{..} \end{..}tokens;\left(, tabular, footnote, …);\if/\fi;\defrecursion (22 shapes);\charcode points (surrogates, >U+10FFFF);\input/\includeof itself.compile_full_project(parse + layout) undercatch_unwindon an 8 MiB thread with a 5 s watchdog. Worker processes (the same binary, re-executed) catch stack overflows and hangs, and the supervisor restarts after the offending case.--minimise),--check, and--diagnostics(time, pages, messages).cargo run --release --example fuzz_compile -- --cases 50000 --jobs 8. For overflow and debug-assert coverage, setCARGO_PROFILE_RELEASE_OVERFLOW_CHECKS=true CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS=true.Fuzz results (50k cases, seed 0xf1a57e40, 5 s watchdog, release with overflow checks and debug assertions)
Bugs: minimised input → before → after
Each row has a regression test that fails before its fix: in
crates/compiler/tests/robustness.rs, incrates/tex-expansion/tests/expand_tests.rs, or in the unit test named in the row.\begin{document}+ 256×\begin{itemize}parser.rs:6972 attempt to add with overflow(count() as u8 + 1)LaTeX Error: Too deeply nested.; depths saturate\setlength{\begin{}$span start must not exceed end(finish_math)\begin{align}a\include{sub}(sub.tex:x\input{sub})cannot merge spans from different documents\count1=99999999999999999999 \advance\count1 by 1tex-expansion/src/expand.rs:3266 attempt to add with overflowNumber too big., then\advancewraps in 32-bit arithmetic with no diagnostic like pdfTeX (-2147483648);\multiply/\divide/\numexpr/\dimexprreportArithmetic overflow.(tex.web §445, §448, §1236-1240);fil+300ls →Illegal unit of measure (replaced by filll).\begin{tabular}{c}(also footnote,\colorbox,\rotatebox,\uline)FlashTeX nesting limit (32) exceeded(an implementation limit below TeX's 255: 255 levels overflow 512 KiB release and 2 MiB debug threads)\loop{\n}(no\repeat)\iterateran to the 2M-step limitRunaway argument? ! File ended while scanning use of \loop.in 0 ms\def\a{\n\a}\ais_known_commandis a set)\def\a#1{\a{#1#1}}\a xTeX capacity exceeded, sorry [main memory size=5000000].in <1 s\AtBeginDocument{\setlength\}}+\begin{document(from page-buildervspace-03.tex)span start must not exceed end(required_group_bounded)span_throughhelper for every opener→end span$$+ 10k\left(…\right)$$TeX capacity exceeded, sorry [grouping levels=255].at the 254th\left, where pdflatex stops\def\a{\csname a\endcsname+ 10k[x 10k]+}\aTeX capacity exceeded, sorry [main memory size=5000000].;\def\b{\b x}\b→[input stack size=10000]\begin{itemize}\item x\@toodeepare counted, not stored\def\a{\ifnum1<2 \fi\a}\a(from a mutated microtype fixture)\if's line is counted only when "Incomplete \if" is reportedReported from other lanes:
vspace-03.texwas bug 9.\looporacle fixtures (microtype, paragraph-layout emergency) were bug 7 rather than quadratic expansion. About 8.5k unsupported-command diagnostics each paid for a full vocabulary scan.c1-ec-10pt-fox-both.texwent from 3.7 s to 0.03 s in release. Across all 48\loopfixtures, the debug parse now takes min 206 ms, median 213 ms, max 532 ms.A follow-up inside this PR: the capacity stops from bugs 8 and 11 are handled like the step limit in the compiler's expansion cache (
c981a008). Without that,expansion_incremental's random-edit test diverged, and the rest of the document was dropped instead of being typeset unexpanded.One unit test changed:
self_referential_macro_hits_explicit_recursion_limitused\newcommand{\loop}{\loop}. The kernel already defines\loop, so that input only reached the step limit through bug 6. It now uses\recurse.Why
A panic kills the worker mid-keystroke, and a hang or OOM freezes the preview. Input that is malformed mid-edit must produce diagnostics, never take the process down. Wording follows TeX where it has a message ("TeX capacity exceeded, sorry [...]", "Number too big.", "Arithmetic overflow.", "LaTeX Error: Too deeply nested.").
Overlap check
gh pr list --state open --search fuzzand--search panic: no fuzz or robustness PRs. render-pipeline: \textsuperscript/\textsubscript raised at sf@size (needs #507 re-pin) #508, pdf: the byte-fidelity oracles find pdflatex, and an absent oracle is visible #289, compiler: soul \so letterspacing and \hl highlight (#502) #513, FT-070: engine performance harness, committed baseline and a >5% regression gate #206 and Integration candidate: kabir-claude queue 2026-09-13b #178 match only on text.--search tex-expansionand--search "fuzz OR robustness OR overflow": nothing touching these functions. compiler: \AtBeginDocument's hook typesets ahead of the body instead of being dropped #499 changes\AtBeginDocumenthandling in the compiler, which is a different code path fromspan_through.coord.py claims:GH-COMPILER-FUZZclaimed by daniel-parent. No other fuzz, panic or robustness claim.vendor/is untouched. The CLI linkscrates/render-pipeline/vendor/compilerandvendor/tex-expansion, so the CLI and render path get these fixes at the next re-pin.Test results
cargo test --manifest-path crates/compiler/Cargo.toml --no-fail-fast: 58 suites, 677 passed, 0 failed, 6 ignoredcargo test --manifest-path crates/tex-expansion/Cargo.toml:test result: ok. 60 passed; 0 failed(expand_tests),ok. 3 passed(incremental_tests),ok. 2 passed(oracle_tests)cargo test --test robustness:test result: ok. 14 passed; 0 failedcargo test --test left_right_nesting:test result: ok. 3 passed; 0 failedNot done
\def\a{\section{\a}}\ain the preamble gives 4.2M in 6.5 s. TeX stops after 100 errors. A cap has to be coordinated with the incremental expander's reuse of diagnostics across checkpoints, so it was left for a separate change.do_endrebuilds@currenviras a String for every\end. Fuzz case 10082 went from 188 s to 3.6 s but still peaks at 2.9 GB.compile_full_project(parse + compiler layout). It does not drive render-pipeline typesetting, font-engine or paragraph-layout, so panics there were not explored.🤖 Generated with Claude Code
https://claude.ai/code/session_012c9XLkHjePPGBuarrmE2mz