Skip to content

exfat: bound contiguous run expansion at cluster heap end - #17

Merged
switch-nz merged 7 commits into
switch-nz:mainfrom
deadp:exfat-run-bounds
Sep 17, 2026
Merged

switch-nz merged 7 commits into
switch-nz:mainfrom
deadp:exfat-run-bounds

Conversation

@deadp

@deadp deadp commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

This branches from #13. Only the last commit (exfat: bound contiguous run expansion at cluster heap end) is new. Once #13 merges, this diff shrinks to that commit alone.

Problem

ExfatFS.chain() with contiguous=True expanded a NoFatChain run into one list element per cluster, where the cluster count came from the untrusted stream-entry length, not the boot sector: a corrupt 2^50 length meant a 2^40-element list(range(start, start + n)) — hang or MemoryError on any read of that file (engine/fs/exfat.py, contiguous branch). The non-contiguous branch was already bounded by its 2 <= c < cluster_count + 2 walk; only the contiguous one was not.

Found by the fuzz corpus: fuzz-findings/exfat-memory-4c12571f.bin replayed the hang. Flip-test substrate from #13 (test_contiguous_run_bounded_by_cluster_heap) pinned the expected bound.

Fix

  • Clamp the contiguous run at the cluster heap end: limit = cluster_count + 2 - start, so a run starting at start ends at byte cluster_offset(start + n) <= cluster_offset(cluster_count + 2). This matches the assertion the Add parser tests, fuzzing, request-gate tests and CodeQL #13 flip test already makes.
  • Report a finding when a run is truncated: "Contiguous run from cluster %d truncated at cluster heap end.", appended only if not already present (server-side FS objects are cached per evidence and runs() is called per stat/read, so a plain append would duplicate the entry in the UI).
  • Expose findings through ExfatFS.info() ("findings": self.findings), mirroring HfsPlusFS — findings reach the UI only via fs.info() in server.py (/api/dir). NtfsFS collects findings but does not surface them through info().
  • Representation is unchanged: still a list(range(...)) of clusters, so all callers (_read_dir_bytes, runs(), _load_bitmap, allocated_extents) behave as before; the mount-time callers (_load_bitmap, allocated_extents) use the non-contiguous branch and can never append this finding.
  • Removed the @unittest.expectedFailure from test_contiguous_run_bounded_by_cluster_heap — it now passes as a normal test.

No changelog entry here by design; CHANGELOG.md lives on the changelog branch (#16) and the entry consolidates there.

Verification

  • Full suite: 289 tests, OK (16 expected failures, down from 17 — the flipped test now passes normally).
  • tests/fuzz.py --replay fuzz-findings/exfat-memory-4c12571f.bin: exits clean, no hang/MemoryError.
  • tests/smoke.py: all checks passed.
  • Patched-image check (same mutation as the flip test): fs.info()["findings"] == ["Contiguous run from cluster 10 truncated at cluster heap end."] (exactly one, stable across repeated runs() calls), and every run satisfies offset + length <= cluster_offset(cluster_count + 2).

Fixes part of #15; test substrate from #13.

deadp and others added 7 commits September 16, 2026 21:52
Locks in the Host, Origin and Sec-Fetch-Site checks that keep other
origins and DNS-rebinding pages from driving the local interface.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Synthetic images are built in code (tests/imagebuild_*.py) so no evidence
or binary fixture is committed. Ten engine bugs found along the way are
recorded as expected failures, each naming the fault: inline data and
inline directories, sparse extents, extent-tree blow-up, the deleted-file
path without a journal, short and truncated hives, over-broad EWF segment
discovery, truncated zlib chunks, and split raw sets.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Volumes are built in code (FAT32 sparsely, as it needs ~34 MB) and opened
the way the server opens them. Seven engine bugs are recorded as expected
failures: scrambled deleted long names and wrong slack for deleted files
on FAT; an unbounded contiguous run list, truncated contiguous directories
and unconverted UTC offsets on exFAT; invalid MFT records served from the
cache, and a record count that ignores volume size when $MFT is unreadable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tests/fuzz.py mutates the synthetic images and drives the app's own entry
points in a child worker, so hangs and memory blow-ups are caught and
their inputs kept for replay. It runs in its own workflow: briefly and
non-blocking on PRs that touch the engine, and at length weekly or on
demand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EwfImage._fh is a method, not a handle, so the harness's cleanup raised
on every EWF case and swamped the real findings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A corrupt stream length expanded a NoFatChain run into one entry per
cluster, unbounded by the cluster count (a 2^50 length means a
2^40-element list: hang / MemoryError on any read). Clamp at the
cluster heap end, report a finding, and flip the expected-failure
test.

Fixes part of switch-nz#15.
@switch-nz
switch-nz merged commit f56e851 into switch-nz:main Sep 17, 2026
25 of 26 checks passed
@switch-nz

Copy link
Copy Markdown
Owner

Merged — reviewed and verified against current main (with #14 and #20): 297 tests OK, expected failures 17 → 16, smoke test passes. The dedupe on the finding and surfacing it through info() are both good calls.

One gap to follow up, not a blocker: the clamp is cluster_count + 2 - start, and cluster_count is the raw u32 from boot sector bytes 92–96, never checked against the volume's actual size. So the fix holds for the saved fuzz input but not for the bug class — corrupt that field as well and the bound goes with it.

Measured by swapping exfat.list for a stub that records len(range(...)) without allocating, on the test_contiguous_run_bounded_by_cluster_heap image with a 2^50 stream length:

cluster_count in boot sector List elements requested
genuine (64) 56
2^28 268,435,448
0xFFFFFFFF 4,294,967,287

All three mount. The last is the same memory exhaustion as before, one 4-byte mutation away, so the fuzzer will likely rediscover it.

Two ways to close it, either sufficient:

  1. Bound cluster_count at mount — clamp to what the volume can hold, (source.size - data_offset) // cluster_size, with a finding when the boot sector claims more. This also protects the FAT-walk branch and every other user of cluster_count.
  2. Stop materialising contiguous runs — represent them as (start, count) as Fuzzer findings: exFAT run-list memory exhaustion, jbd2 AttributeError, NTFS record-count hang #15 suggested, so no length allocates anything.

(1) is the smaller change; (2) removes the allocation class entirely. I've noted on #15 that item 1 is partly fixed rather than done.

switch-nz added a commit to deadp/strata that referenced this pull request Sep 17, 2026
Known issues pointed only at switch-nz#15, which covers crashes and hangs. The
bugs that matter more to an examiner reading release notes are the ones
that return wrong or missing results with no finding, tracked in switch-nz#19:
exFAT timestamps labelled UTC without applying the offset, sparse ext4
files at the wrong offsets, misread ext4 inline data, scrambled deleted
FAT names and misplaced FAT slack, a truncated EWF chunk returned short,
and split raw sets loading as a partial disk. Each is still an expected
failure on main, so each is listed individually rather than behind a
link.

0.1.0 is not tagged yet, so what has merged since this changelog was
written ships in it: the EWF read bounds (switch-nz#14), the exFAT contiguous
run clamp (switch-nz#17) and the parser tests, fuzzing and CodeQL (#13) are
added. Raw/dd is described as single-file and split raw sets move to
Not implemented, matching the README correction in switch-nz#20.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@deadp
deadp deleted the exfat-run-bounds branch September 17, 2026 03:45
switch-nz pushed a commit that referenced this pull request Sep 17, 2026
The #17 heap-end clamp in chain() trusts cluster_count itself, so a
boot sector claiming 0xFFFFFFFF clusters (offset 92) still let a
NoFatChain stream expand by the billions: the fuzzer's stat loop hung
on the mutated image even after that fix. Bound the boot sector's
cluster count by what the image actually holds past the cluster heap
(with a finding), which protects both the contiguous limit and the
FAT-walk bound.

Fixes part of #15.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants