Skip to content

PDF: batch native path evidence - #24

Merged
myhloli merged 3 commits into
mainfrom
codex/rust-pdf-stage19-visual-evidence
Sep 29, 2026
Merged

myhloli merged 3 commits into
mainfrom
codex/rust-pdf-stage19-visual-evidence

Conversation

@myhloli

@myhloli myhloli commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Stacks on PDF: batch text visibility and reuse visual evidence #22 / Stage 18 634aabe; private protocol advances 25 → 26.
  • Adds a Rust PDFium path-evidence bridge that traverses all PATH leaves once, reads/transforms subpaths once, reuses draw state, and produces both drawing-line candidates and PDFPathInfo.
  • Preserves Python reference fallback for old extensions, ABI mismatches, and nonstandard pages. Binding-level math.hypot keeps stroke-width floating-point results bit-for-bit equivalent.
  • Adds real hit/fallback diagnostics and optional visual-evidence stage profiling.

Correctness

  • Full 32-PDF / 299-page public replay matched Stage 18 for ModelJson, MiddleJson, assets, and diagnostics.
  • Native path evidence was force-compared with the Python reference page by page across all 32 PDFs / 299 pages; line and path-info outputs were identical. Native hit count: 299, fallbacks: 0.
  • MinerU dev@ee87e8e: 31 eligible Flash text outputs and 32 medium shared-chain outputs all matched Stage 18.
  • Rust/session: 5474 passed, 1 skipped.
  • Python/legacy: 4769 passed, 706 skipped.
  • Cargo workspace tests, Clippy -D warnings, rustfmt, Ruff check, and changed-file format checks passed.
  • ABI3 wheel verified in an isolated CPython 3.14 environment; auto and Rust outputs matched, protocol 26, and source/wheel extension SHA-256 both equal f6dcdb5654cd30c3617aecbc22fd57ec3c1a49b8aa6d1e13ef2e3df062ec0ab2.

Performance

Formal measurements use one warmup plus five hot runs per document, paired baseline/candidate execution, and isolated process-tree RSS sampling.

Chain Stage 18 Stage 19 Reduction
DocVortex public parse, 32 PDFs 15.484642 s 15.202684 s 1.82%
MinerU Flash text, 31 PDFs 15.335368 s 15.045106 s 1.89%
MinerU medium shared, 32 PDFs 15.531999 s 14.566413 s 6.22%

Gates:

  • Public parse maximum time ratio: 1.0020; maximum RSS ratio: 1.0330; no sample exceeded the 5% sustained time gate.
  • Flash maximum time ratio: 1.0266; maximum RSS ratio: 1.0165.
  • Medium maximum time ratio: 1.0411 (below the reverse-retest trigger); maximum RSS ratio: 1.0142.
  • All compared outputs were equal.

Diagnostics show _extract_page_paths_and_lines() fell from 0.940791 s to 0.129739 s (86.2%) in the matched cProfile run. The original 2× performance target remains incomplete.

Evidence: docs/rust-pdf-stage19-evidence.md and output/pdf/native-kernel-20260929-stage19/ locally.

Not done

  • Merge, GitHub release, PyPI publication, and downstream release synchronization are not performed.
  • Stage 20 should focus on PDFPageTextGeometry.chars materialization (about 80.5% of the profiled visual-evidence time), followed by native text-snapshot page/textpage ownership.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T13:45:21.077739Z 732152f New commits
🔒 Security Review ✅ Completed 2026-09-29T11:34:03.888739Z be0b1a1 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be0b1a1db2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread rust/docvortex-pdfium/src/path_evidence.rs Outdated
Comment thread rust/docvortex-pdfium/src/path_evidence.rs Outdated
@myhloli
myhloli changed the base branch from codex/rust-pdf-stage18-evidence to main September 29, 2026 13:05
@myhloli
myhloli merged commit c9a4e94 into main Sep 29, 2026
21 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 732152ffb0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

let rounded = subpath
.points
.iter()
.map(|&(x, y)| ((x * 1000.0).round() / 1000.0, (y * 1000.0).round() / 1000.0))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use Python-compatible rounding for open-path corners

When a transformed coordinate is exactly halfway at the third decimal, Rust's round() uses ties-away-from-zero while Python's round(value, 3) uses ties-to-even. For example, the open filled path (0,0) → (10,0) → (10,0.062) → (0,0.0625) passes the Python corner-set and axis-alignment checks but is rejected here because 0.0625 becomes 0.063 instead of 0.062, so the native backend omits a drawing line emitted by the reference backend. Apply Python's rounding semantics to both the actual and expected corner sets.

Useful? React with 👍 / 👎.

unsafe fn color(raw: *mut c_void, getter: ColorFn) -> Rgba {
let (mut red, mut green, mut blue, mut alpha) = (0, 0, 0, 255);
if getter(raw, &mut red, &mut green, &mut blue, &mut alpha) == 0 {
return (0, 0, 0, 255);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve unknown fill colors instead of fabricating black

When FPDFPageObj_GetFillColor returns failure for a filled path, the Python reference treats the alpha as opaque only for visibility but stores fill_rgba=None; this fallback instead publishes (0, 0, 0, 255). On malformed or unsupported color states, consumers such as _path_has_visible_nonwhite_fill() consequently treat an unknown fill as a confirmed black background and can create spurious code-block evidence. Keep the visibility fallback separate from the optional RGBA value.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant