fix(loader): stream temp dir during cleanup to avoid OOM - #322
Open
arimu1 wants to merge 1 commit into
Open
Conversation
File.listFiles(FilenameFilter) materializes every /tmp entry before filtering, which can exhaust heap when hundreds of thousands of unrelated directories exist. Use Files.newDirectoryStream with a jansi-* prefix filter so only matching entries are visited. Fixes fusesource#300
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
File.listFiles(FilenameFilter)inJansiLoader.cleanup()withFiles.newDirectoryStreamand a prefix filter forjansi-{version}-*entries.File.listFilesinternally callsFile.list(), which materializes every name in/tmpinto aString[]before the filter runs — this causesOutOfMemoryErrorwhen/tmpholds 250k+ unrelated entries (reported in JansiLoader.cleanup() throws OutOfMemory error if /tmp/ has 250,000+ directories #300 with-Xmx16m).Files.newDirectoryStreamapplies the filter while iterating directory entries, so only matchingjansi-*files are visited and unrelated temp entries are never loaded into heap.Fixes #300
Test plan
mvn test -Dtest=JansiLoaderTest(Java 21)cleanupDeletesStaleJansiLibsWithoutListingAllTempEntriespasses: stale lib removed, locked lib kept, 1000 decoy files untouched