Skip to content

ext4: preserve sparse file offsets and zero unwritten extents on read - #458

Merged
deitch merged 1 commit into
diskfs:masterfrom
illusionfield:fix/ext4-sparse-read
Nov 5, 2026
Merged

deitch merged 1 commit into
diskfs:masterfrom
illusionfield:fix/ext4-sparse-read

Conversation

@illusionfield

Copy link
Copy Markdown
Contributor

Problem

File.Read in the ext4 driver walked the extents as if they formed one contiguous
byte stream. A file with a hole (a logical block range without an extent) therefore
returned data from the wrong offset, and after #430 any read that touched a hole
failed with read offset ... falls outside extent. Uninitialized extents
(ee_len > 32768) returned whatever the disk blocks contain instead of zeros.

Change

Read now resolves the requested logical byte range against the extent map:

  • holes and uninitialized extents read as zeros, and their disk blocks are not fetched;
  • extents must be sorted, non-overlapping and must not overflow the backend; a
    violation is reported as a corrupt mapping instead of being read through;
  • on a short or failing backend read the exact byte prefix, the file cursor and the
    backend error are preserved;
  • the loop stops at the first extent past the window (extents are validated to be
    sorted);
  • a backend that returns io.EOF together with a complete read, as io.ReaderAt
    allows, is treated as success.

No writer, extent parser or dependency changes. Write keeps the check from #430.

Tests

filesystem/ext4/file_sparse_read_test.go adds:

  • a byte-oracle contract test over holes, unwritten extents and extent boundaries,
    including seeks;
  • backend-fault tests (failure on the first and on a later extent, io.EOF with a
    full read) with a fault-injecting backend.Storage;
  • an invalid-mapping test (unsorted, overlapping, overflowing extents);
  • a test proving holes and unwritten extents never reach the backend;
  • a real-image regression: mkfs.ext4 -d from a sparse source, an unwritten extent
    added with debugfs fallocate, garbage written into the unwritten blocks, verified
    with e2fsck -f -n. It skips when mkfs.ext4, debugfs or e2fsck is missing,
    so the macOS and Windows CI legs are unaffected.

With the previous Read restored, all seven new tests fail.

Docs

filesystem/ext4/ext4.md documents the read semantics for holes, uninitialized
extents and extent ordering.

Verification

  • go test ./filesystem/ext4/ on Linux with Go 1.25 and e2fsprogs 1.47.2: ok
  • golangci-lint run ./... (v2 config, 0 issues), go build and make vet on
    macOS, GOOS=windows cross build and lint: ok

Read logical ranges instead of concatenating physical extents. Holes and
unwritten extents read as zeros without touching the backend, the exact
byte prefix, cursor and backend error are preserved on short or invalid
reads, and overlapping, unsorted or overflowing mappings are rejected.
Sorted extents let the read loop stop at the first extent past the
window, and a backend that reports io.EOF together with a complete read,
as io.ReaderAt allows, is treated as success.

Add independent byte-oracle, seek, unwritten-boundary and backend-fault
regressions, plus a real-image regression built with mkfs.ext4 -d from a
sparse source and extended with an unwritten extent by debugfs; it skips
when e2fsprogs is not installed. Document the sparse read semantics in
ext4.md. No writer, extent parser or module dependency changes.

@deitch deitch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Let CI rip.

@deitch
deitch merged commit 044eda3 into diskfs:master Nov 5, 2026
39 of 40 checks passed
@illusionfield
illusionfield deleted the fix/ext4-sparse-read branch November 5, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants