Skip to content

feat: end a stream's data where the file does, keep none of it, and leave a guard's cut to the guard - #297

Merged
AdCodicem merged 146 commits into
mainfrom
ccr-6d6c0ee6-dqm4lm-174
Oct 25, 2026
Merged

AdCodicem merged 146 commits into
mainfrom
ccr-6d6c0ee6-dqm4lm-174

Conversation

@AdCodicem

@AdCodicem AdCodicem commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

What this changes

Batch 16 of M02's step 4: where a stream's data ends, what of it the reader keeps, and what a guard's cut leaves.

Why

Closes #170
Closes #174
Closes #220
Closes #224
Closes #230
Closes #263

The issues' shapes measured on a76c9da with a survey of the corpus's 35,716 confirmed streams. The maintainer
settled the batch's eleven questions as recommended, then the questions of the reviews (R3-18, replaced by R4-1; R4-2,
R4-15, R4-23; R6-1 to R6-3; R8-1 to R8-3; R12-1 to R12-3). Thirteen adversarial review rounds, 325 findings, each put
to a refuter, then a coverage pass whose verdicts were each put to a skeptic; every major fixed in the batch with a
test that failed before its fix. The journal of 2026-10-25 in docs/status.md lists them all.

Filed under M02:

#264's first gap, a /Prev through an object a guard cut, is paid here; its second stays for batch 19.

Definition of done

The project's definition of done applies to every change:

  • Unit tests cover the behavior, including its degenerate and hostile cases — 643 new, 4,689 to 5,332 (7,060 to 7,704 with the remote documents)
  • Anything an independent tool should confirm is covered by the integration suite — 3,688 against qpdf, 3,682 passed and 6 skipped, twice
  • The build has no new warnings, and any suppression is local and justified
  • docs/status.md records the batch: the journal of 2026-10-25 and the figures at a glance.
  • Public API carries XML documentation, and the site under docs/website matches what now exists — 263 pages built, the site's check clean
  • dotnet format has been run

Anything reviewers should look at closely

Merge: squash, at the maintainer's choice (145 commits are too many for a rebase merge, and main takes no merge
commit). The title is feat:, since the batch adds PdfReaderLimits.MaxStreamLength, so that semantic-release reads
the squashed commit as a feature.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T

claude added 30 commits October 23, 2026 05:28
A stream read from a file kept a copy of its raw data for as long as the stream was
held, by the object cache or by the caller. Streams whose data share a stretch of the
file each kept their own: two hundred of them held 200 MB from a file of 1 MB, and
validating three hundred nested object streams held 300 MB, whatever the object stream
budget let go.

The data is read each time it is asked for now, and kept by nothing: a source in memory
lends it where it lies, and any other reads it into an array of its length, handed back
whole on a short read rather than copied again.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
A stream's encoded data was bounded by the file alone. Decoding handed back data no
filter decodes - none named, an image filter, one the library does not know, Flate over
data that is none - whole, past MaxDecodedStreamLength and unreported, and a /Length
between what an array holds and what an int does threw an OutOfMemoryException each
time the data was read.

PdfReaderLimits.MaxStreamLength bounds what is read of one stream at once, 256 MB by
default, under the code limit.stream-length: past it the data is cut, marked cut by a
guard, so that what the filters lose past the cut is no damage, and the guard is
reported once for each stream, or thrown under ThrowOnLimit. Data decoding hands back as
it is falls under it, and MaxDecodedStreamLength keeps bounding what filters produce. A
length past what an array holds ends at the guard, whose ceiling is the array's. The
stream's length stays the one the file holds. The manifest can raise the new limit.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…herever its window ends

A stream's length was misjudged three ways. More than four bytes of white space before
endstream, or a comment, made a right /Length a wrong one. A /Length the file cannot hold
grew the window to MaxObjectLength even when an endstream was found: it read 21.5 MB for
five bytes, reported limit.object and marked the object cut. And the search in the window
took the first endstream it held, past the object's endobj and the next header: 49 bytes
of object 6 for a stream of five.

A declared length is confirmed by endstream after any white space and comments, the file
read on as far as they run. Every other length - missing, unusable, past the file, or one
endstream does not follow - goes to one search: the bytes the parser holds, then the
file, up to the nearer of the first object header and the next object the index places,
once for each stream. A stream's data never counts toward MaxObjectLength, and no window
grows to find its end.

With no endstream before that stop, the object's endobj plays its part: a declared length
endobj follows is kept, and otherwise the endobj that ends the object before the stop ends
the data, its end-of-line left out. Without one, a declared length that ends before the
stop is kept, and the data otherwise runs to the stop, a declared length past it cut
there. The parser alone applies the same rules to its buffer.

A cross-reference stream whose /Length the chain could not read is searched as the
window's search did, a carriage return kept and no header stopping it, now past the
window rather than through one MaxTrailerLength bounds.

Tests that declared a length past the next object, or cut stream data with
MaxObjectLength, expect what the decisions give: the data ends at the object's endobj or
at the next object, and stream data is cut by MaxStreamLength. PDFBox's pdfbox3947 object
stream 417 keeps its declared 1,302 bytes, inside which its Flate data ends.

Refs #174, #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
An object that lost both its closing delimiter and its endobj had its array or dictionary take in
the next object's header and value, up to that object's endobj, and read as ending with one: the
missing endobj went unreported, and a trailer that lost its >> took the next update's entries.

Inside an array or a dictionary, where an element, a key or a value is expected, the next object's
header now ends every container left open. A header is the one the search for a stream's
endstream stops at: N G obj, digits alone, N from 1 to the largest object number and G within the
generation range, white space alone between them, and obj a whole keyword a window's edge does not
reach. The cut is reported once, at the innermost container, as "An object header ended the
object", and the object lacks its endobj. Containers skipped past the depth the parser follows end
there too. An object stream's member is never followed by a header, and a value alone is no
container: neither applies the rule.

Its cost: a closed container holding text that reads as a header ends there, and its endobj is
judged missing. None of the 344,361 objects of the corpus holds one.

Refs #230

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…s are read

A cross-reference stream whose /Length the chain cannot read takes its data up to the first
endstream. Rows that spell one cut the data short: the section was read in part and called
malformed, and an update's objects past the spelled rows read as the revision it supersedes. Data
with no endstream at all ran to the end of the file and was read as rows.

Where the section's own row for the /Length lies among the rows the data holds, the object it
places is read there, nothing kept or reported, and the data is taken again to the length it
gives when an endstream confirms it, then decoded once more. A row that does not stand leaves the
data as the chain took it, and the length is checked once the chain is read, as before.

Data with no endstream before the next object the sections read so far place, or the end of the
file, makes its section malformed: the file's fault, and no guard's.

The search past the window this relies on came with the single search for a stream's end: a
stream of 70,000 bytes of rows reads as its direct twin, with no limit.trailer, and
MaxTrailerLength bounds the dictionary alone. Tests hold that too.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…e file's

A /Length naming an object a guard cut was judged by what the cut left of it: null, reported as
a /Length that holds no length, or an integer the cut parted, 1 of 15, reported as a declared
length the data contradicts. Neither is a value the file gave.

The answer is keyed on the object the reference leads to, through a chain of references too, and
never on the value read: one a guard cut takes a form of its own, and one a guard left out of the
index the existing one. An integer read before the cut is only a place to look for endstream at,
never kept as a length; the data is otherwise found by its endstream, and nothing is said of the
length. A missing endstream is still the file's, and reported. A cross-reference stream's
/Length read once the chain is read is left alike. No message is made for a length not judged.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… index

A /Filter, a /DecodeParms, or a value of a step's parameters that led to an object a guard cut, or
left out of the index, read as no filter, or as "/Flate" of "/FlateDecode" parted: an object
stream decoded so read its header as stray bytes, the catalog it held was looked for elsewhere,
and a sound file under a lowered limit was judged broken. The chain of filters now ends at such
a value, as at the element past MaxFilterChainLength: the steps before it run, the data is left
encoded, and an object stream so left is hidden and its members taken as the guard's.

A guard that loading a filter's value reached was taken for a cut of the stream's own data,
which hid the stream's own faults, and made the reader and the validator disagree as read order
went. Decoding now says whether the stream's own data is cut, which is what the readers of rows
and members ask.

While the chain is read, a cross-reference stream's /W, /Index, /Size, /Filter or /DecodeParms
naming an object MaxObjectLength cuts made the section malformed, and the index was rebuilt. The
section is now the guard's: the chain ends there, nothing is rebuilt, and the guard is reached
once the chain is read, thrown at opening when the caller asks.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…eader's limits

ADR 34 gains the amendment of 2026-10-23: MaxStreamLength is a new guard; EndStreamSearchPasses
stays an internal constant, its reason rewritten; the bound on the gap before endstream is gone;
EndObjLookahead is a first read's size; MaxTrailerLength bounds a cross-reference stream's
dictionary alone; a value read through a reference to an object a guard cut or left out of the
index is the guard's. The architecture, the threat model and the reader-limits pages say the
same.

Refs #170, #174, #220, #224, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… otherwise

The referee held that the reader and qpdf both took object 417's data of PDFBox's pdfbox3947 up
to object 441's endstream, past a 32 KB block of zeros that erased its own: inside its window the
reader did not bound its search. Its single search for a stream's end now stops at the next
object the index places, in the bytes the reader holds as past them: the reader stops inside the
block and keeps the declared 1,302 bytes, where qpdf, whose search no object bounds, reads 34,152.

The expectation is corrected, not weakened: the stream joins the known disagreements, for qpdf's
reason, as PDFium's urban planning report's object 695 did, and the remarks no longer say the
reader's search is unbounded inside its window. The manifest's expectations of the document hold.

Refs #174

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… each time

The reader-limits concepts, the guides to the reader's limits and its memory, and the
diagnostics reference say what MaxStreamLength bounds, what limit.stream-length reports, and
that a stream's raw data is read from the file each time it is asked for, kept by nothing.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…he references

The diagnostics, object-model and validation-rules references, the concepts of diagnostics, lazy
reading and validation, the threat model and M02 describe the stream-end rules, the values a
guard cut or left out of the index, and the header rule. The XML documentation of the codes,
the pipeline, the reader, PdfStream, Decode and the endobj rule says the same.

Refs #174, #224, #230, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…er its window ends

The review of the batch found four places where a stream's end depended on how it was read rather
than on its bytes.

- The search for a stream's endstream is kept once per stream, but whether the object's endobj
  follows the declared length was kept with it. A rebuild that replaced the /Length then read the
  stream with the old answer. It is now read again whenever the declared end changes, and only
  then, so a second reading of an unchanged stream reads nothing more.
- An endstream the window held, with the window ending before what follows it, left the object's
  endobj unseen. Past the window, what follows is read on in the file. It is now read on in both
  cases, so where the window ends decides nothing.
- A file that ends on the stream keyword, or on its carriage return, read the stream as no data
  and said nothing: the parser took the end of the bytes for a window too short to say where the
  data starts. The end of the file says it: the data starts there and holds nothing, and the
  missing endstream is reported, as after a line feed.
- Closing the reader released the searches but not the checks made past white space after a
  length.

The states a missing endstream ends in, and the next object a fault names, are documented as the
search now decides them.

Refs #174

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… keep no length from it

A /Length that leads through a reference to an object a guard cut is read as the guard's: what the
cut left is only a place to look for endstream at. The review of the batch found two places where
the length's fault still said otherwise.

- The fault named the object the /Length refers to, not the one the guard cut at the end of the
  chain: "names object 20 0, which one of the reader's limits cut" of an object read whole.
- The integer the cut left, a parted 1 of 15, was kept as the fault's declared length. It is now
  only where the parser looks for endstream first, and the fault declares no length.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… an object it cut

A cross-reference stream's /W, /Index, /Size, /Filter or /DecodeParms that leads to an object
MaxObjectLength cuts became the guard's in this batch. A /Prev or /XRefStm written as a reference to
such an object was left out. The section it names was reported missing as the file's fault, the
index was rebuilt, and the validator said no section placed the object, though the newer section
did.

Now the chain ends there for the guard, as for the other values. Nothing is rebuilt, the older
sections and their trailers are the guard's to hide, and the guard is reached once the chain is
read: reported, or thrown at opening under ThrowOnLimit. A hybrid file's stream so hidden ends the
chain with it, since the older sections' rows would stand for its own.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…no object it ends in a trailer

Two faults of the header rule, which ends an object's open containers at the next object's header.

- Inside a dictionary or an array, a header the window's edge cut short ("6 0", "6 0 ob") was
  read for what it seemed: its number and generation as keys that are not names, or the number
  as the value of the key before it. A larger window drops these reports, but where none follows,
  at MaxObjectLength or in the rebuild's trailer scan, they were kept as the file's faults. The
  parse now counts as having met the edge there, as it does for a reference the edge cut.
- The report said the header ended "the object" in a trailer, or a value read alone, where no
  object is open. It now says the header came inside the construct left open.

A test case now holds the dictionary's value path, where a header stands for a key's value.

Refs #230

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…eferences and its guards

The review of the batch found four ways a /Length the chain could not read was settled wrongly.

- Only the first link of a chain of references was read while the chain was. A /Length object
  holding "6 0 R", where object 6 waited for the chain or was cut by MaxObjectLength, was judged to
  hold null, and the guard was never reached. The whole chain is followed now, and the length waits
  for the chain wherever a link cannot be read.
- Taking the data again to the section's own row for the /Length used the first row numbering it,
  a row of a reserved type included, which reading the rows ignores. It now uses the row the reading
  keeps.
- When MaxObjectLength cut that /Length object, the data was not taken again, and the rows past
  the endstream they spell were charged to the file: the section was said to lack them, and an
  older revision's rows stood for them. The integer the cut left is now where an endstream is
  looked for. When none confirms it, those rows are the guard's, as rows past a decoding guard are,
  and the chain ends there.
- Data MaxStreamLength cut was still checked against the length once the chain was read, the
  guard's cut reported as the file's. Whether a guard cut it is known only once decoding read it,
  and it is checked then.

Refs #220, #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…axStreamLength

The helper that opens a corpus document as the acceptance tests open it mapped every reader limit a
manifest entry can raise but the new MaxStreamLength. A document whose entry raises it would have
been read under the default by the stream length tests, and under its own bound by the others.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…it can be

The review of the batch found two places where a comment, which ISO 32000-1 (7.2.3) counts as white
space outside a stream's data, was read wrongly at the end of one. Both rules were settled with
the maintainer.

- A /Length too short that ends on a "%" of the data: the rest of the line read as a comment before
  the endstream, which confirmed the wrong length, and the rest of the data was lost with no report.
  The same comment let an endobj after it keep the short length when the stream has no endstream.
  A comment now counts there only once white space comes before it: a "%" right where the length
  ends is the data's, the length goes unconfirmed, and the search finds the real end.
- A stream with no endstream ends at its object's own endobj when only white space follows it up
  to the next object. A comment there kept the endobj from ending the data: the short length was
  kept, or the data took in the endobj. Comments after the endobj are white space now.

The endobj is followed forward through the reads of the search for endstream, which reads every
byte up to where it stops already, rather than read back from there: a comment can only be told
from the start of its line. The bytes before the stop are no longer read a second time.

Refs #174

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…lter past every guard

HasImageFilter says whether Decode hands back an encoded image, and read none of the filters'
parameters to tell it. Since a value of a step's parameters that leads to an object a guard cut
ends the chain where it is read, a Flate step before a DCT one whose /DecodeParms a guard cut
left its data Flate-encoded: HasImageFilter still said it was the image, and an extractor that
trusts it would have written Flate data as a JPEG.

It now reads what decoding reads before the image filter, as the maintainer settled: the
parameters of a Flate or LZW step that comes before it, and nothing for a chain that ends sooner.
A parameter held in an object stream decodes that stream first, as decoding the stream would.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…the index

A /Length that leads to an object a guard left out of the index is unknown, and not judged. With
it, the endstream the stream lacked went unreported too: the data was taken where its object's
endobj ends it, and neither the diagnostics nor the validator saw a fault. A /Length a guard cut
already reported it as the file's fault. As the maintainer settled, both are reported alike now:
"The stream's /Length names object 21 0, which one of the reader's limits left out of the index,
and no endstream follows its data ...". The object named is the one the chain of references leads
to, as for a cut one.

A message about a stream's length is now made only when it is reported: a stream parsed again,
or one whose length is read once the chain is, made each message and dropped it.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…nnot end its data

A cross-reference stream whose /Length the chain cannot read takes its data up to the first
endstream; with none before the next object or the end of the file, its section is malformed. A
/Length that the newer section places, but that MaxObjectLength cuts, waits for the chain the same
way, and was judged the same: the fault said no section placed the object, the index was rebuilt,
and the validator reported a malformed section. Without the guard the length is read, and the
section with it.

As the maintainer settled, such a section is the guard's, as one whose /W, /Index or /Filter a
guard cut is: it counts as cut, the chain ends there, nothing is rebuilt, and the guard is reached
once the chain is read, thrown at opening under ThrowOnLimit. The fault names the reference the
guard cut, through a chain of references too.

Refs #220, #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… passes to Decode

Decode promises the diagnostics a caller passes of their own what decoding meets, each time the
stream is decoded, and limit.decoded-stream reaches them so. The cut MaxStreamLength makes went to
the document's diagnostics alone, once, as the reader read the data: a caller who passed their own
got data cut short with no report, and could not tell, the cut being marked only internally.

As the maintainer settled, those diagnostics now receive limit.stream-length with each decoding
that meets the cut, and the document's still hold it once. A test of ASCII85 data the guard cut
added the two lists together and expected the guard once in all: it now finds it once in each,
the expectation following the settled rule. The message is made once, as the reader reports it
once, rather than each time the data is read past the bound.

The XML documentation of Decode and of PdfStreamData.GetBytes says what the guard does to the data
and where it is reported.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
A stream's raw bytes taken from a document opened over memory are a view of that memory, not a
copy: they keep it alive after Dispose, for as long as they are held, and change as it does.

Refs #170

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…s decoding does

HasImageFilter now reads the parameters decoding reads before the image filter, and loading one
past MaxObjectLength reaches the guard: under ThrowOnLimit, the caller is told so, as Decode would
tell them.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… and HasImageFilter

Decoding loads the values its filters read, and so can reach a guard under ThrowOnLimit, or
meet a disposed document: the exceptions of Decode and HasImageFilter now say so.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
The batch's patch left lines of where a stream's data ends that no test reached. Each now has a
file that reaches it:

- an endobj that ends a longer token, read in one read or parted between two, or right where the
  data starts;
- a stream parsed alone, in a window that is not the end of the data, with no endstream in it;
- a container skipped past the depth the parser follows, which the next object's header ends;
- a comment in the gap after a length that runs past the window and the first read after it, and
  the check the reader keeps for the stream's next reading;
- "endstreamx" read on past the window, and white space read on to the end of the file;
- a stream left unsearched whose declared length runs past the end of the file.

The construction of streams nested in one another's dictionaries is shared by two tests now.

Refs #174

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… endstream is found

The place a deferred cross-reference stream's endstream is looked for after is where its data
starts plus the bytes the chain took, never before the start of the file.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…as they may

Once the document's searches for endstream have read as much of the file as they may, white space
after a declared length that runs past the window is not read on. A length the first read past the
window cannot confirm is kept without a search, and one whose endstream that read ends after
leaves what follows the keyword unseen. Neither path had a test.

Refs #174

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
A cross-reference stream whose /W gives its type field no width makes every row of type 1. The
section's own row for its /Length, before rows that spell an endstream, then places it, and the
data is taken again to the length it gives. No test had such a section.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…ta again

A newer table that refuses its row for the object a cross-reference stream's /Length names leaves
no row of that stream standing for it: the data stays as the chain took it, up to the endstream
its rows spell, and the section is said to lack the rows past it. No test had such a chain.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
claude added 22 commits October 25, 2026 07:21
…letes

Data a proven endstream ends past white space under a hidden /Length takes in white space the
length would leave out: a Flate stream short of its checksum's last byte read a space or a line
feed in its place, and reported a checksum that disagrees, a warning, where read whole it is only
short. A checksum that ends in that white space says nothing of the file, as one it leaves short.

The decoders' docs no longer say such data decodes as read whole everywhere: LZW or RunLength data
that lacks its end marker takes the white space in.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…e precedes it

A cross-reference stream is read past any white space its offset names, as its header is found by
reading its object. A classic table was told by the first 32 bytes alone: named 29 or more bytes
before its keyword, which they held only in part or not at all, it was not read, the index was
rebuilt, and startxref was said to name neither the keyword nor an object. The probe now reads on
past white space, comments and a token its edge may cut, as far as a trailer's window reaches.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…gain knowing its end hidden

Data decoded again knowing its end hidden held no more rows than the decoding before it, which
was meant for a decoding MaxDecodedStreamLength stopped. A decoding no guard stopped, its end taken
for the file's, drops a predicted row the data parts, which the second decoding undoes as far as it
goes: the rows it completes, before the endstream the rows spell, were left to the guard, and the
page whose row it held read as null where the whole read and the direct twin read it. The cap now
holds where a guard stopped the decoding before only.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…n that reached it

The proof that an endstream ends a stream's own data takes the first significant byte past the
keyword from the read-on that found it, which tells whether an endobj follows, only where one did.
Past an endstream the data spells, no endobj after it, it read the white space again from 16 bytes,
doubling: thirty streams under a cut /Length, each spelling one before 64 KB of white space, had
the searches read close to four times the file, where three are said. They read it about twice.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… end of the file

The probe that looks for a classic table's xref keyword reads on past white space as far as
MaxTrailerLength reaches. No test named more white space than that, nor white space that runs to
the end of the file: a bound reached and read again, a loop, would have passed the suite. The guard
is now held reached, reported or thrown, and white space to the end names no table.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…ong whatever it holds

The subsection one of whose elements a guard cut is the file's fault where the element read is
wrong whatever the cut one holds. No test held the edges: a count of 0, valid, a first number of
2147483647, valid with a count of 0 or 1, a first number past it and a count past the object
numbers, nor the messages such a fault is said with. Each is now held, and a mutant of each edge or
message fails.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…tever a cut value holds

A cross-reference stream a value's cut keeps from being read was the guard's unless its widths or
its numbering, as far as read, were wrong. Data whose length the file gives, decoded whole, may hold
fewer rows than the counts read declare however few bytes the widths read allow a row: three bytes
under a /Size of 8. Read whole, the section lacks rows, the file's, and the index is rebuilt; it was
taken for the guard's, the chain ended there and the catalog was lost, or thrown under ThrowOnLimit.
Such a section is now the file's to lack rows, there and not looked for nearby, and the index is
rebuilt. Data a guard cut or whose end it hides, a /Length the chain cannot read yet, and filters a
reference holds say nothing. The test that held the guard's verdict on three bytes holds 56 now.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…ngth may hold

Data an endstream proven its own ends under a /Length a guard hid decoded to its end, every
filter's end report silenced, a Flate checksum silenced where the data ended in white space, and
a predicted row undone wherever a filter of the chain ran out. That silenced a wrong checksum whose
bytes all lie before the white space, reported one a comment completes, decoded white space and a
comment's digits as LZW codes or hex, faults the whole read never meets, undid rows behind ASCIIHex
that ended where its format says, and charged the file with an object stream's last member the
end-of-line the search left out had shortened.

The first filter now decodes the data as taken, and where it ends it as its format says with
nothing to report, that is the data read whole. Otherwise it decodes it again up to where the run
of white space and comments that ends it starts: what it meets before is the file's, running out
there says nothing, and where its output may lack what the run holds, Flate's tail or LZW and
RunLength codes, the data's end is the guard's, as cut, for the filters after it, the predictor and
an object stream's reader. The filters after a first that ended whole decode as read whole.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… holds beside a cut

The architecture, the diagnostics reference and the threat model said data an endstream proven its
own ends past white space decodes as read whole, every filter's end silenced: its first filter now
decodes it again up to the run of white space and comments when it does not end it as its format
says, its output's end the guard's where it may lack what the run holds, and the filters after a
first that ended whole report as read whole. The known gap is narrowed to match.

The lists of what is the file's beside a value a guard cut gain data that holds too few rows
whatever the value holds, in ADR 34, the architecture, the concepts and the references; the reader
limits reference no longer calls a subsection of no rows a fault. The search costs no longer claim
about twice the file for three ways a deferred stream's white space crosses three times, the lazy
reading page counts them as the others do, and the probe's comments, its regrowth among the known
gaps, and the scope of a value's cut whatever the type are said as the code does them.

Refs #224, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…as read whole

A cross-reference stream beside a value a guard cut whose data cannot hold the rows declared was
taken for a section that cannot be read: the chain ended there, its trailer and the older ones were
not merged, so /Info, /ID or /Root only they give were lost, the validator judged no trailer value
of it, and at a place a few bytes from the offset named it was passed over for an older revision's
stream nearby, which was read in its place. Read whole, it is read and lacks rows. It now is so
under the guard too: its trailer merged, the chain going on to its /Prev, the index rebuilt, and
taken where it was moved to. A width past what an int counts no longer wraps the smallest row a
section's widths allow, and a lone byte or row is said as one.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…nt no data holds, may lie

The first filter under a hidden end decoded the data as taken and, where it found anything to
report, again up to the run of white space and comments that ends it. A % the data starts with
was taken for a comment and the whole data dropped; in ASCII85, where a % is a digit, a line
starting with one was dropped and not marked cut; spaces after LZW codes spelled an end code
taken for the data's own; the chain's Flate predictor ran on an ASCIIHex step; raw deflate, or
a zlib checksum past bytes that look like a comment, lost its last bytes to the guard; and data
of white space alone under ASCIIHex was marked cut.

A comment now starts only past white space within the data, and ends no ASCII85 data. Flate data
whose deflate body ends in the data as taken is whole, its repair reported, a checksum that
disagrees only where it reaches into the run silenced. LZW, RunLength and ASCII data decode up
to the run, their end markers counting only there, and the predictor runs on Flate and LZW steps
alone. Data that is all white space under ASCII85 or ASCIIHex is whole.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…tream parts

A cross-reference stream whose /Length only its own rows place is taken to the first endstream,
and its rows may spell one. The section's own row for that /Length is read from the rows held,
but where the spelled endstream parts the predicted row that holds it, the data, decoded as data
whose end the file gives, dropped that row: the length was not found, the section was charged
with the rows it lacked, and a valid file's index was rebuilt. The data is now decoded once more
knowing its end hidden, its parted row undone as far as it goes, and the row read from there; what
that decoding reports is dropped with it.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…side a cut reads whole

The architecture, the references, the concepts, the threat model and the XML docs said data a
hidden length ends decodes as read whole where its first filter ends it with nothing to report.
They now say what the code does: the first filter decodes up to the run of white space, and of
comments past white space within the data but in ASCII85, and Flate data whose deflate body ends
in the data as taken is whole. A cross-reference stream lacking rows beside a value a guard cut is
said to be read as read whole, its filters written direct, rather than unreadable. A deferred
length's row in a predicted row a spelled endstream parts, the search costs past the window, and
the comments the concept pages left out are said as well.

Refs #220, #224, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… a guard hid

A cross-reference stream whose data cannot hold the rows its values declare, beside a value a
guard cut, was read as read whole: its trailer merged and the chain going on. The rows its data
held, read with the cut value, were not read, and an older section's rows stood for them, so a
superseded catalog was served. The cut value was never loaded, so the guard was neither reported
nor thrown under ThrowOnLimit, and the validator charged the file with a wrong /Size the hidden
rows would have set right.

The cut value is now loaded once the chain is read. Where the data may hold a row at the fewest
bytes a row may take, the section is also the guard's: the chain ends there, as where rows a guard
cut, the size rule leaves it unjudged and the validator says it went unchecked. Data that holds
no row leaves the chain going on.

Refs #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…tream parts, guard kept

A cross-reference stream whose /Length only its own rows place is taken to the first endstream its
rows may spell. Where that endstream parts the predicted row holding the row for the /Length, the
data was decoded once more knowing its end hidden only when the row lay past the rows the first
decoding held. A PNG predictor whose first row is parted leaves the data as decoded, and the TIFF
predictor leaves a parted row differenced: the row was read garbled, and a valid file's index was
rebuilt. That decoding also dropped a decoding guard it reached, which ThrowOnLimit threw: the
rows past the bound were charged to the file in one mode and thrown in the other.

The data is now decoded once more wherever a subsection numbers the object and no row standing for
it was found, and the row looked for in the rows that decoding holds. A guard that decoding
reaches stands, and where it found no row the rows past the bound are the guard's, as where the
first decoding reaches it. A test holds data that holds no whole row, which no row is looked for in.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
Two rules of the hidden end had no test that fails without them. Flate data whose deflate body ends
in the data as taken is whole only where decoding stayed within MaxDecodedStreamLength: past it,
the guard is reported and the data cut, as read whole. Data that is all white space and comments
before its endstream decodes to nothing, cut by the guard, under LZW, Flate and RunLength.

Refs #224

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…uard's

The architecture, ADR 34, the references, the concepts and the threat model said a section whose
data lacks the rows its values declare beside a value a guard cut reads as read whole, the chain
going on. They now say the guard is reached once the chain is read, and that rows the data may
still hold, read with the cut value, are the guard's: the chain ends there and the validator says
them unchecked. A deferred length's row is said to be found in any predicted row a spelled
endstream parts, a decoding guard that search reaches standing. A comment cut short is completed,
a missing checksum and a first number below 0 are named, and two tests are cited.

Refs #220, #224, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
… far the guard lets rows read

The last round of mutants found two rules of the deferred length's second decoding that no test
held. The data is decoded again only where a subsection numbers the object the /Length names: one
no row numbers is never looked for there, and a decoding guard that search would reach is not
reported. Where that decoding stops at MaxDecodedStreamLength short of the /Length's row, the rows
it holds are read as far as the guard lets them, a page's row in the parted predicted row among
them, not only those the decoding before held. The RunLength chain's files share their predicted
rows' writing.

Refs #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
Remote corpus run 78 failed one test, the deferred length checked on data MaxStreamLength cuts past
every row, at a bound of 29 bytes. Its bounds were those of the Flate data this container's zlib
writes at CompressionLevel.Optimal; the runtime Microsoft ships, as CI installs it, bundles zlib-ng,
which writes other bytes, so 28 of them held fewer rows. The same test binary, run under that
runtime here, fails as CI did, and passes under the distribution's.

The file now holds its Flate data in a stored block for that test, the bounds 49, 50 and 60 counted
from its layout, which no compressor moves. The test that bounds the decoded data keeps the
compressed form, which does not depend on it. The whole unit suite passes under both runtimes.

Refs #170, #220

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…s or direct calls

A coverage run of the unit suite on the committed corpus left 39 of the lines the batch changed
partial or missed. Each one now has a test. A document takes the line where one reaches it; where
no file can, the test calls the internal member directly. The new tests cover:
- a rollback to a mark that records no decoding reports;
- a range a source in memory lends nothing;
- a window's edge outside the file;
- a place outside the file said to end data beside white space;
- data the file lost under the document;
- a deferred length's chain of references one longer than a parse resolves;
- sections that spend the endstream searches;
- a section beside a cut /W whose data may hold rows;
- ASCII85 white space under a hidden end.

What no input takes stays partial: a null check on a set that is never null there, an operand a
caller's precondition makes true, conditions that contradict each other, and the arms the
compiler adds. Two faults the tests met are filed rather than pinned: a /Prev naming an object
whose header a guard cut rebuilds the index once the chain is read, and a deferred length is
reported past the chain's endstream once the searches are spent.

Refs #170, #174, #220, #224, #263, #292, #293

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
…tion it records

The test of a token the cross-reference probe met cut said the probe saw "xref" and the table's
own window refused "xrefs". The probe now reads on past a token its edge reaches, so it reads
"xrefs" whole, finds neither the keyword nor an object, and the table never sees it. The comment
says so, and the test now holds the section the probe records: not found, the fault naming
neither. Where MaxTrailerLength stops the probe right after "xref", the table's window refuses
the token under another rule, which is filed.

Refs #174, #295

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
The journal of 2026-10-25 records the batch:
- the issues' shapes measured and the eleven questions settled with the maintainer;
- the thirteen adversarial reviews and the decisions they asked for;
- the coverage pass and the faults it filed;
- what the batch did for its six issues, the tests and mutants, and the checks;
- the debts it filed, and the maintainer's decision on the remote corpus's hosts.

The figures at a glance follow the branch: tests, coverage and the lines left with why no input
reaches them, CI and the remote corpus's runs 75 to 80, and the measurements of reading every page
and of a wrong /Length. Step 4's list is brought up to date, with the fourteen debts placed in the
batches they join.

Refs #170, #174, #220, #224, #230, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
@codecov

codecov Bot commented Oct 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.67638% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/AdCodicem.Pdf/IO/Filters/PdfFilterPipeline.cs 98.70% 0 Missing and 1 partial ⚠️
src/AdCodicem.Pdf/IO/PdfObjectParser.cs 99.62% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Owner Author

Remote corpus run 81, on b9df537, failed only at its last step, "Report documents that could not be fetched", for a cause outside this PR. Two documents could not be downloaded from web.archive.org:

  • remote/canada/livecycle-es10-ircc-imm1344-certified-dynamic-xfa.pdf: Connection refused;
  • remote/us-states/powerbi-pdfium-acrobat-covid-dashboard-nine-updates.pdf: TLS handshake timed out.

Every test over the 240 other documents passed: 7,763 acceptance tests and 3,745 referee checks, none failed. This is the hosts' failure that #204 records. The fix decided today for it (the cache, more patient retries, and the copy of their own in #291) is not written yet, and none exists to port. I am re-running the job once.


Generated by Claude Code

@AdCodicem AdCodicem changed the title fix: end a stream's data where the file does, keep none of it, and leave a guard's cut to the guard feat: end a stream's data where the file does, keep none of it, and leave a guard's cut to the guard Oct 25, 2026
The maintainer squashes the pull request, its 145 commits too many for a rebase. The record says so,
gives the check of each commit as far as it had gone, and adds run 81 of the remote corpus. Run 81
lost two documents to web.archive.org at its first attempt and passed whole at its one rerun.

Refs #170, #174, #220, #224, #230, #263

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PiFf9bygtdCAcN4va1ss8T
@AdCodicem
AdCodicem merged commit ce30948 into main Oct 25, 2026
22 checks passed
@AdCodicem
AdCodicem deleted the ccr-6d6c0ee6-dqm4lm-174 branch October 25, 2026 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment