Bound heap usage when digesting signed attachments - #385
Merged
Conversation
WSS4J's AttachmentContentSignatureTransform#processAttachment digests each signed attachment (cid: reference) by calling mark(Integer.MAX_VALUE) on the attachment source stream, reading it to the end and calling reset(), so the attachment stays readable afterwards. The file backed streams handed out so far (NonBlockingBufferedInputStream) honour that mark by buffering everything read after it on the heap, so the peak heap scales with the attachment size (a 250 MB attachment peaks at ~390 MB) although the content is already spilled to a temporary file. This affects the signing and the verification side alike. Add com.helger.phase4.util.MarkableFileInputStream, which implements mark/reset by re-positioning the underlying FileChannel, and hand it out at all the file backed provider sites: - WSS4JAttachment.createOutgoingFileAttachment (File and byte[] overloads) - WSS4JAttachment.createIncomingFileAttachment (both overloads) - SoapHeaderElementProcessorWSS4J (decrypted attachment temporary file) - AS4IncomingHandler._createReadMultipleISP Digesting then runs in constant heap - a 250 MB attachment completes at -Xmx64m, where the previous stream throws an OutOfMemoryError - and the 2 GB limit of heap buffering mark/reset (see the WSS4J comment) no longer applies to file backed attachments; a 2.5 GB attachment was verified at -Xmx64m. The digest input is unchanged, only the carrier of the same bytes is swapped. In-memory attachments (<= 64 KB) keep their byte array streams, which already support constant heap mark/reset. Not covered: incoming attachments that are encrypted as well as signed. There WSS4J digests its own CipherInputStream, which does not support mark/reset, so AttachmentContentSignatureTransform wraps it in a BufferedInputStream itself and buffers the plaintext on the heap. That needs a separate change on the decryption path.
shunkica
force-pushed
the
issue-380-markable-file-input-stream
branch
from
August 9, 2026 12:13
586ed51 to
a3c44e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #380
Problem: WSS4J digests signed attachments by calling
mark()/reset()on their streams, forcing the entire attachment into heap memory even when using temporary files for large payloads.Solution: Introduced
MarkableFileInputStream, a new stream implementation that performsmark()/reset()viaFileChannelpositioning instead of heap buffering. This is used at all file-backed attachment sources: outgoing files, compressed temp files, and incoming decrypted attachments.Small attachments (≤64 KB) continue using byte-array streams unchanged. Signatures remain identical, only the carrier changes.
Testing: All existing tests pass (129 in phase4-lib, 200 in phase4-test). New tests verify mark/reset behavior and round-trip integrity. Probe on a 250 MB attachment with
-Xmx64m: previously failed withOutOfMemoryError, now completes in 181 ms with identical digests.Notes: Encrypted-and-signed attachments still heap-buffer the plaintext during digestion (separate mechanism, out of scope). See PR body for limitations.