Repository navigation
Add per-reader readahead control - #172
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #172 +/- ##
==========================================
+ Coverage 60.03% 60.11% +0.08%
==========================================
Files 82 82
Lines 13060 13082 +22
Branches 2331 2334 +3
==========================================
+ Hits 7840 7864 +24
- Misses 4280 4281 +1
+ Partials 940 937 -3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
nclack
marked this pull request as ready for review
September 28, 2026 18:24
Rename the C flag to enable_readahead and apply it to GDS file descriptors before cuFile registration. Cover reader-to-executor propagation, cached and reopened files, and failed advice cleanup and retry in forced compatibility mode.
Expose CMAKE_CUDA_ARCHITECTURES as a Docker build argument and pass it to both the native and editable Python builds. Select 120-real for the RTX 5070 Laptop GPU on auk while retaining the portable image default. Validated a Debug+coverage image on auk: native build 21.3s, editable install 23.9s, all 39 CTest tests passed, Python import smoke passed, and both assembly objects contain sm_120 code. Workflow lint and git diff --check passed.
Wait for callback progress on an independent condition variable and establish waiter readiness under the scheduler lock. Keep a 30-second CTest watchdog for deadlocks. Validated 100 repeated scheduler runs, delayed callbacks that fail the old test and pass the new test, and a stalled callback that fails through the watchdog.
github-actions Bot
added a commit
that referenced
this pull request
Oct 6, 2026
Scattered crops can benefit from disabling OS file readahead, but scans can slow down substantially. Add `FileReader(readahead=False)` so applications can choose per reader. The default remains `True`, preserving existing behavior. The option applies to buffered bulk reads with both CPU and CUDA executors. Linux uses `POSIX_FADV_RANDOM`; macOS uses `F_RDAHEAD=0`. Each newly opened chunk or shard gets the setting, including after cache eviction. A rejected OS request fails the read with an I/O error. Metadata readers remain independent; this option does not clear or bypass the page cache. C callers get an additive `damacy_file_reader_create_with_config` constructor; the existing constructor and legacy `Config` adapter retain OS-default readahead. Validation: - Linux CPU: 27/27 CTest checks passed; macOS: 28/28; CUDA: 39/39. Each suite includes Python tests. - Coverage includes per-reader isolation, default C constructor compatibility, cached and reopened files, failed advice and retry, argument validation, and CPU/CUDA output correctness with readahead on and off. - Release build, documentation build, ThreadSanitizer, and coverage checks passed. - Python lint/format checks and `git diff --check` passed. --------- Co-authored-by: Nathan Clack <nclack@biohub.org> 3ccabe4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scattered crops can benefit from disabling OS file readahead, but scans can slow down substantially. Add
FileReader(readahead=False)so applications can choose per reader. The default remainsTrue, preserving existing behavior.The option applies to buffered bulk reads with both CPU and CUDA executors. Linux uses
POSIX_FADV_RANDOM; macOS usesF_RDAHEAD=0. Each newly opened chunk or shard gets the setting, including after cache eviction. A rejected OS request fails the read with an I/O error. Metadata readers remain independent; this option does not clear or bypass the page cache.C callers get an additive
damacy_file_reader_create_with_configconstructor; the existing constructor and legacyConfigadapter retain OS-default readahead.Validation:
git diff --checkpassed.