Repository navigation
exfat: bound contiguous run expansion at cluster heap end - #17
Conversation
Locks in the Host, Origin and Sec-Fetch-Site checks that keep other origins and DNS-rebinding pages from driving the local interface. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Synthetic images are built in code (tests/imagebuild_*.py) so no evidence or binary fixture is committed. Ten engine bugs found along the way are recorded as expected failures, each naming the fault: inline data and inline directories, sparse extents, extent-tree blow-up, the deleted-file path without a journal, short and truncated hives, over-broad EWF segment discovery, truncated zlib chunks, and split raw sets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Volumes are built in code (FAT32 sparsely, as it needs ~34 MB) and opened the way the server opens them. Seven engine bugs are recorded as expected failures: scrambled deleted long names and wrong slack for deleted files on FAT; an unbounded contiguous run list, truncated contiguous directories and unconverted UTC offsets on exFAT; invalid MFT records served from the cache, and a record count that ignores volume size when $MFT is unreadable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tests/fuzz.py mutates the synthetic images and drives the app's own entry points in a child worker, so hangs and memory blow-ups are caught and their inputs kept for replay. It runs in its own workflow: briefly and non-blocking on PRs that touch the engine, and at length weekly or on demand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EwfImage._fh is a method, not a handle, so the harness's cleanup raised on every EWF case and swamped the real findings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A corrupt stream length expanded a NoFatChain run into one entry per cluster, unbounded by the cluster count (a 2^50 length means a 2^40-element list: hang / MemoryError on any read). Clamp at the cluster heap end, report a finding, and flip the expected-failure test. Fixes part of switch-nz#15.
|
Merged — reviewed and verified against current One gap to follow up, not a blocker: the clamp is Measured by swapping
All three mount. The last is the same memory exhaustion as before, one 4-byte mutation away, so the fuzzer will likely rediscover it. Two ways to close it, either sufficient:
(1) is the smaller change; (2) removes the allocation class entirely. I've noted on #15 that item 1 is partly fixed rather than done. |
Known issues pointed only at switch-nz#15, which covers crashes and hangs. The bugs that matter more to an examiner reading release notes are the ones that return wrong or missing results with no finding, tracked in switch-nz#19: exFAT timestamps labelled UTC without applying the offset, sparse ext4 files at the wrong offsets, misread ext4 inline data, scrambled deleted FAT names and misplaced FAT slack, a truncated EWF chunk returned short, and split raw sets loading as a partial disk. Each is still an expected failure on main, so each is listed individually rather than behind a link. 0.1.0 is not tagged yet, so what has merged since this changelog was written ships in it: the EWF read bounds (switch-nz#14), the exFAT contiguous run clamp (switch-nz#17) and the parser tests, fuzzing and CodeQL (#13) are added. Raw/dd is described as single-file and split raw sets move to Not implemented, matching the README correction in switch-nz#20. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The #17 heap-end clamp in chain() trusts cluster_count itself, so a boot sector claiming 0xFFFFFFFF clusters (offset 92) still let a NoFatChain stream expand by the billions: the fuzzer's stat loop hung on the mutated image even after that fix. Bound the boot sector's cluster count by what the image actually holds past the cluster heap (with a finding), which protects both the contiguous limit and the FAT-walk bound. Fixes part of #15.
This branches from #13. Only the last commit (
exfat: bound contiguous run expansion at cluster heap end) is new. Once #13 merges, this diff shrinks to that commit alone.Problem
ExfatFS.chain()withcontiguous=Trueexpanded a NoFatChain run into one list element per cluster, where the cluster count came from the untrusted stream-entry length, not the boot sector: a corrupt 2^50 length meant a 2^40-elementlist(range(start, start + n))— hang orMemoryErroron any read of that file (engine/fs/exfat.py, contiguous branch). The non-contiguous branch was already bounded by its2 <= c < cluster_count + 2walk; only the contiguous one was not.Found by the fuzz corpus:
fuzz-findings/exfat-memory-4c12571f.binreplayed the hang. Flip-test substrate from #13 (test_contiguous_run_bounded_by_cluster_heap) pinned the expected bound.Fix
limit = cluster_count + 2 - start, so a run starting atstartends at bytecluster_offset(start + n) <= cluster_offset(cluster_count + 2). This matches the assertion the Add parser tests, fuzzing, request-gate tests and CodeQL #13 flip test already makes."Contiguous run from cluster %d truncated at cluster heap end.", appended only if not already present (server-side FS objects are cached per evidence andruns()is called per stat/read, so a plain append would duplicate the entry in the UI).findingsthroughExfatFS.info()("findings": self.findings), mirroringHfsPlusFS— findings reach the UI only viafs.info()inserver.py(/api/dir).NtfsFScollects findings but does not surface them throughinfo().list(range(...))of clusters, so all callers (_read_dir_bytes,runs(),_load_bitmap,allocated_extents) behave as before; the mount-time callers (_load_bitmap,allocated_extents) use the non-contiguous branch and can never append this finding.@unittest.expectedFailurefromtest_contiguous_run_bounded_by_cluster_heap— it now passes as a normal test.No changelog entry here by design;
CHANGELOG.mdlives on thechangelogbranch (#16) and the entry consolidates there.Verification
tests/fuzz.py --replay fuzz-findings/exfat-memory-4c12571f.bin: exits clean, no hang/MemoryError.tests/smoke.py: all checks passed.fs.info()["findings"] == ["Contiguous run from cluster 10 truncated at cluster heap end."](exactly one, stable across repeatedruns()calls), and every run satisfiesoffset + length <= cluster_offset(cluster_count + 2).Fixes part of #15; test substrate from #13.