Skip to content

PDF: own textpages and streamline geometry materialization - #27

Merged
myhloli merged 2 commits into
codex/rust-pdf-stage19-visual-evidencefrom
codex/rust-pdf-stage20-geometry-materialization
Sep 29, 2026
Merged

myhloli merged 2 commits into
codex/rust-pdf-stage19-visual-evidencefrom
codex/rust-pdf-stage20-geometry-materialization

Conversation

@myhloli

@myhloli myhloli commented Sep 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Advance the private native protocol from 26 to 27 while preserving public auto|python|rust compute and auto|legacy|session render selection.
  • Add a Rust-owned PDFium textpage load/count/read/close bridge for native snapshots, with ABI validation and explicit fallback to the existing reference path.
  • Reduce NativeTextSnapshot.geometry() materialization overhead by indexing shared font dictionaries, caching field names, constructing bbox inputs directly, and sharing origin/loose/tight coordinate objects without changing public geometry shape or mutability semantics.
  • Add parity and real textpage lifecycle coverage; record validation and performance evidence in docs/rust-pdf-stage20-evidence.md.

Correctness

  • Public replay: 32 PDFs / 299 pages, two candidate runs per document; ModelJson, MiddleJson, asset hashes, and diagnostics matched Stage 19 exactly.
  • MinerU Flash: 31 eligible PDFs, all complete outputs equal.
  • MinerU medium shared: 32 PDFs, all complete outputs equal.
  • Rust/session full tests: 5475 passed, 1 skipped.
  • Python/legacy full tests: 4769 passed, 707 skipped.
  • Cargo workspace tests, Clippy -D warnings, rustfmt, Ruff check, and changed-file formatting passed.
  • ABI3 wheel verified in a CPython 3.14 isolated dependency environment; source and wheel extension SHA-256 both 77c16ff6c926daea4b8ec631f192ba0f3593791cd885391cbe13d4cdf6d862d8. DocVortex core and MinerU Flash smoke outputs matched for auto and rust.

Measured performance

One warmup plus five hot runs per document; MinerU variants alternate by document; RSS uses isolated process-tree sampling.

Chain Stage 19 Stage 20 Reduction Max first time Max first RSS
DocVortex public parse, 32 PDFs 14.966399 s 14.923734 s 0.29% 1.06446 1.00480
MinerU Flash text, 31 PDFs 14.827657 s 14.759856 s 0.46% 1.02284 1.00549
MinerU medium shared, 32 PDFs 14.738283 s 14.601869 s 0.93% 1.07333 1.12712

All first-run >5% time regressions passed reverse-order retests. The single first-run >5% shared RSS regression retested at 1.02646 and was not sustained. Full outputs were equal in every benchmark.

Focused profiling reduced prepare_visual_evidence() from 0.703 s to 0.612 s across the 32-PDF source collection profile; the eight-document geometry stage decreased from 0.257527 s to 0.226503 s (12.0%). The Rust-owned textpage path executed 161/161 focus snapshots with no fallback, though _extract_owned_text_snapshot() remained approximately 0.529 s, showing that textpage wrapper creation itself was not the major raw-snapshot cost.

Not done

  • Original 2× target remains incomplete.
  • No merge, GitHub release, version bump, PyPI publication, or downstream dependency synchronization.
  • MinerU is integration-only; its current dev@f504cff worktree and untracked examples/ were not modified.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T13:07:31.926581Z d504c5e PR opened
🔒 Security Review ✅ Completed 2026-09-29T13:07:24.059607Z d504c5e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@myhloli
myhloli merged commit 0552b0c into codex/rust-pdf-stage19-visual-evidence Sep 29, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant