Skip to content

Add read-only EROFS support with LZMA decompression - #454

Merged
deitch merged 41 commits into
diskfs:masterfrom
carbonin:erofs
Nov 3, 2026
Merged

deitch merged 41 commits into
diskfs:masterfrom
carbonin:erofs

Conversation

@carbonin

Copy link
Copy Markdown
Contributor

Add an EROFS reader and integrate detection into Disk.GetFilesystem().

The implementation uses the EROFS on-disk documentation, particularly the core format specification. Compression structures follow the kernel’s EROFS
on-disk definitions
, with decoding based on the MicroLZMA format documentation.

Features covered

  • Superblock parsing, CRC32-C checksum verification, volume labels, and filesystem block sizes from 512 bytes through 64 KiB.
  • Compact and extended inodes, directory traversal, regular files, and hard links.
  • File metadata: permissions, setuid/setgid/sticky bits, timestamps, UID/GID, inode numbers, link counts, and encoded device numbers. Device nodes, FIFOs, and sockets are exposed as metadata.
  • Uncompressed FLAT_PLAIN and FLAT_INLINE layouts, including reads spanning external data and inline tails.
  • LZMA/MicroLZMA compression with full and compact indexes, large physical clusters, and logical clusters larger than filesystem blocks.
  • Packed-inode fragments, including fragment tails and files stored entirely as fragments.
  • Shifted and interlaced incompressible extents within compressed layouts.
  • Go io/fs access, sequential reads, seeking, and random reads. Each open file caches one decoded extent and implements safe concurrent ReadAt calls, serialized per open file.
  • Symlink inspection: metadata identifies links, and reading a link returns its stored target.

Features not covered

  • Image creation or filesystem mutation; existing images return read-only errors.
  • LZ4, DEFLATE, or Zstandard decompression.
  • Chunk-based files, including chunk-based deduplication and sparse chunks; multi-device images.
  • Compressed inline tails, partial compressed references, and alternative compressed extent metadata. Fragment support does not imply support for every deduplication encoding.
  • 48-bit filesystem layouts, metaboxes, or compressed metadata.
  • Extended attribute retrieval, including ACLs, capabilities, and SELinux labels. Inline xattr space is skipped when locating file data; long and image-share xattr prefixes are unsupported.
  • Symlink resolution, including links in intermediate path components.
  • Directory blocks larger than the filesystem block size or unrecognized feature flags.

Limits and resource management

Compressed extents are limited to 1 MiB encoded and 12 MiB decoded.
Compact indexes support logical clusters up to 16 KiB with 4-byte records, or 4 KiB with 2-byte records.

The caller manages the backend’s lifetime and must keep it open and unchanged while using the filesystem. FileSystem.Close() closes neither the backend nor open file handles. Close file handles separately to release their cached data, then close the backend after use.

Tests

Tests include reproducible erofs-utils 1.9.4 fixtures, generated images, io/fs conformance, malformed metadata and compression indexes, filesystem bounds, read-only errors, cache reuse, and concurrent reads.

Motivation

assisted-image-service currently extracts nmstatectl from RHCOS root filesystems and packages it into boot ramdisks for minimal installation images. Its existing EROFS extraction path invokes dump.erofs, parses directory listings, and requires custom-built erofs-utils and xz/
liblzma.

This reader enables the service’s migration to native Go extraction from LZMA-compressed root.erofs images embedded in rootfs.img CPIO archives, avoiding shell commands and temporary extraction for this path and allowing those native build and runtime dependencies to be removed.

Add support for reading EROFS files that use LZMA compression. Parse EROFS
compression configuration and map logical file ranges through both compact
and full compressed indexes. Handle big physical clusters, shifted and
interlaced plain clusters, packed fragment tails, whole-file fragments, and
compressed data stored in a packed inode.

Decode EROFS MicroLZMA data using the existing xz dependency, requesting
the exact logical extent size so partial reads and final clusters decode
correctly. Validate compression algorithms, configuration fields, index
values, extent boundaries, and packed-inode references before reading data.
Unsupported compressed layouts fail with descriptive errors instead of
returning invalid data.

Add an integration test that generates an EROFS image with mkfs.erofs and
reads it entirely through the native reader. The image uses LZMA, 1 MiB
physical clusters, and packed fragments to exercise the compressed index
and fragment paths against formatter output. Use a deterministic,
mostly incompressible payload because small or repetitive data can be
stored inline and would not reliably produce the compressed layout under
test. The large payload spans cluster boundaries, and its zero-filled tail
encourages fragment packing. Skip the test when mkfs.erofs is unavailable.
Directory names are bounded by each entry's nameoff and the following
entry's nameoff. The final entry has no following entry, so
parseDirectory treated its name as extending to the end of the directory
block. When an image padded the final name with NUL bytes, that padding
became part of the parsed name and triggered the embedded-NUL validation,
causing otherwise valid directories to be rejected.

For the final entry, stop at the first NUL after its nameoff when one is
present. Keep the existing end-of-block fallback for unpadded names, and
continue using the next entry's nameoff for all earlier entries.

Update the ordering test to place names after the complete dirent table so
it exercises the intended multi-entry directory layout.
This removes the need to run rerun parseSuperblock after getting the
checksum information from the inital call. Now we can parse the
superblock, get all the information we need and separately run the
verification.
EROFS flat-inline inodes store complete blocks in external data blocks and
the final partial block in the inode. readInodeData calculated the
non-inline size, but reads beginning in external data used the full request
length for the external read. Requests crossing the boundary therefore
tried to read inline-tail bytes from the external-data region, which could
return the wrong data or reject a valid image as out of bounds.

Cap the external read at the non-inline boundary. If the request still has
remaining bytes, advance the destination and offset, then read those bytes
from the inline tail using the existing inline-data path.

Add a regression test for a 4,097-byte flat-inline file. It verifies that a
full read returns all 4,096 external-data bytes followed by the final byte
from the inode’s inline tail.
Add two EROFS 1.9.4 images generated with fixed block size, ownership,
timestamp, UUID, and a deterministic 96 KiB payload. Document the source
tree, payload generator, mkfs commands, and image SHA-256 values.

The uncompressed test opens the committed image, enumerates the root
directory, reads an inline symlink target, resolves a nested path, and
verifies FLAT_PLAIN file contents by SHA-256. The compressed test reads
the same nested payload from an LZMA compact-index image using 64 KiB
clusters and a packed fragment tail, comparing every byte.

These fixtures provide compatibility coverage in normal CI when
mkfs.erofs is unavailable; the generated-image tests remain optional
local validation.
Add a deterministic 8 KiB LZMA fixture whose 96 KiB file uses the full
compressed-index layout without the whole-file packed special case. Reading
it complements the existing compact-index fixture and verifies exact payload
reconstruction through the full-index decoder.

Extend the committed fixtures with in-memory corruption cases. For compact
indexes, verify rejection of a Head2 record with an unavailable secondary
algorithm, out-of-image extents, invalid fragment offsets, and an invalid
packed-inode NID. For full indexes, verify rejection of an unadvertised
compression algorithm, Head2 without a secondary algorithm, out-of-image
extents, and malformed LZMA data.

Recompute the superblock checksum after each mutation so the tests exercise
the compressed-index validation paths rather than checksum verification.
No reader behavior changes; this adds regression coverage for existing
untrusted-image validation.
Opening a directory previously exposed the on-disk "." and ".." entries,
always labeled the directory ".", and returned the full entry list every
time ReadDir was called with a non-positive count. Filter dot entries,
preserve the opened path's base name in Stat, and make ReadDir(n <= 0)
return and consume only the remaining entries, matching fs.ReadDirFile
semantics.

Run fstest.TestFS over the uncompressed fixture and add checks for invalid
paths, missing paths, directory reads, and write-mode opens.

Also add regression coverage for 255-byte and padded final directory names;
compressed ReadAt and Seek operations at block, fragment, cluster, last-byte,
and EOF boundaries; and metadata for block and character devices, FIFOs,
sockets, symlinks, hard links, device IDs, inode numbers, and link counts.
Compact directory inodes store their link count in i_nb, not high
block-address bits. Keep the link count but clear i_nb before calculating
the start block so external directory data is read from its valid on-disk
location.
When compression configuration is enabled, available_compr_algs is an algorithm
bitmap. Reject bits outside the four defined algorithms so invalid metadata
fails with a clear error instead of being silently ignored.
Clamp positive ReadDir counts to the remaining entry count before adding
the cursor. A partially consumed directory can now accept math.MaxInt
without overflowing an int or panicking while allocating the result.

Add a regression test for ReadDir(1) followed by ReadDir(math.MaxInt),
including the subsequent EOF result.

Validation: go test ./filesystem/erofs -run '^TestReadDirLargeCount$' -count=1
Expose the original EROFS/Linux new_encode_dev value through
StatT.DeviceID. Combining decoded major and minor values as major<<8|minor
overlapped their fields and made distinct devices indistinguishable when
the minor exceeded 255.

Document how callers can recover the major and minor components. Cover
character and block devices in compact and extended inodes, including the
reported collision, major 256, minor 256, and both maximum fields.

Validation: go test ./filesystem/erofs -run '^TestDeviceIDPreservesEncoding$' -count=1
Treat nonempty ReadAt calls at or beyond EOF as ordinary end-of-file
conditions. Keep negative offsets invalid, and return success for empty
buffers at nonnegative offsets, including offsets beyond EOF.

Cover empty and nonempty files, exact EOF, past EOF, math.MaxInt64,
zero-length buffers, negative offsets, and reads crossing EOF.

Validation: go test ./filesystem/erofs -run '^TestFileReadAtEOF$' -count=1
Small File.Read and ReadAt calls previously fetched and decoded the entire
containing LZMA extent on every call, including fragments read through the
packed inode. Streaming a large binary through a 32 KiB buffer consequently
allocated and decoded the same multi-megabyte output hundreds of times.

Retain one decoded LZMA extent per open File, identified by inode number and
logical byte range. Check that range before mapping or fetching another
extent, and pass the same cache through packed-inode fragment reads.
Replace the entry when another extent is decoded and release it on Close.
The retained entry is bounded by the existing 12 MiB decoded-extent limit;
encoded extents remain capped at 1 MiB. Decoder dictionaries and other
temporary allocations are separate from this retained-cache bound. No
global cache or image-wide accumulation is introduced.

Guard the cursor, cache, and file lifecycle with a per-file mutex so
concurrent ReadAt calls cannot race while populating or replacing the
entry. Calls on the same handle serialize; independently opened handles
have independent caches. ReadAt continues to leave the sequential cursor
unchanged. The backend must remain unchanged while the reader is in use.

Add deterministic tests that alternate Read and ReadAt, revisit cached
bytes, and assert one compressed-data fetch for both ordinary and packed
fragment extents. Add concurrent ReadAt coverage and a streaming benchmark
that checks every output byte against its source. Document the benchmark
command and the cache's ownership, bounds, and concurrency behavior.

Paired local measurements of BenchmarkLargeExtentStreaming, with the same
benchmark copied into the uncached parent snapshot for comparison:

  go test ./filesystem/erofs -run '^$' \
    -bench BenchmarkLargeExtentStreaming -benchmem -benchtime=1x -count=1

  Buffer       State     ns/op          B/op           allocs/op
  32 KiB       before    67411243557    6190148792       74848694
  32 KiB       after       563528306      45627520         539187
  15 MiB       before      493610839      45606816         538764
  15 MiB       after       527805379      45606864         538764

For 32 KiB streaming this reduces elapsed time from 67.411 s to 0.564 s
(about 120x faster) and allocated bytes from 6.19 GB to 45.63 MB (about
136x fewer). Small-buffer extraction now has approximately the same
decoding cost as the whole-file buffer. The whole-buffer case remains
comparable rather than gaining the repeated-decoding improvement.

The synthetic payload is 15 MiB, built from 64 KiB records containing
16 KiB of deterministic pseudo-random bytes followed by 48 KiB of zeros.
mkfs.erofs -zlzma -C1048576 -Efragments generates large compressed extents
and a packed tail. Each timed iteration opens a fresh file with an empty
decoded cache, reads and verifies all bytes, then closes the file. Image
generation, filesystem initialization, and read-buffer allocation are
outside the timed loop. B/op is cumulative allocation, not peak memory.

Environment: Linux amd64, Intel Core i7-10850H, Go
go1.26.10-X:nodwarf5, erofs-utils 1.9.4, GOMAXPROCS=12. These are single
local observations; the host page cache was not cleared. They demonstrate
the buffer-size scaling improvement on this synthetic workload and are
not general performance guarantees.

Validation:
- go test -race ./filesystem/erofs -count=1
- paired BenchmarkLargeExtentStreaming runs shown above
Remove extent-end calculations that were unconditionally overwritten by
the final scan, and drop the unused clusterOffset argument from
compressedExtentEnd. The scan remains the single source of the extent end.

Validation: go test ./filesystem/erofs -count=1
Drop inode format, blocks, and dotOmitted fields that do not affect
validation or public metadata. Keep format interpretation local to parsing.
Reject unsupported compressed inline tails while parsing the header instead
of retaining an otherwise unused inlineDataSize field.

Validation: go test ./filesystem/erofs -count=1
Describe byte offsets, filesystem blocks, logical clusters, packed-inode
offsets, compact index packing, and MicroLZMA's properties-byte conversion.
Reference the on-disk definitions and kernel MicroLZMA documentation without
tying the explanations to a particular tool or dependency version.

Only comments change. Validation: gofmt and git diff --check.
Enumerate unsupported compressors, layouts, metadata extensions, inline
tails, and partial compressed references in the public package documentation.
Clarify that accepting the shared fragments/dedupe feature bit does not
imply support for every deduplication encoding.

Validation: git diff --check.
State that Stat exposes a symlink object and Open/ReadFile read its stored
target name. Path traversal does not resolve symlinks, including intermediate
components. Direct binary extraction therefore requires the regular-file
deployment path.

Validation: git diff --check.
Document that Read borrows a backend which the caller must keep open and
unchanged. FileSystem.Close is a no-op and closes neither the backend nor
open files. File.Close releases the per-file decoded extent; callers close
their backend separately.

Validation: gofmt and git diff --check.
Return ErrReadonlyFilesystem from directory, node, link, permission,
ownership, timestamp, rename, and remove mutations, matching SetLabel,
OpenFile write flags, and File.Write. Creation of a new image remains
unimplemented, as documented separately.

Validation: go test ./filesystem/erofs -count=1.

@deitch deitch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good. I am glad to have it in; thank you. I adding EROFS support to linuxkit a few years back as it looked like it made sense; no idea why I did not add it here.

Some small requests about artifact generation and documentation, and a question. Other than that, looks good.

Comment thread filesystem/erofs/testdata/README.md Resolved
Comment thread filesystem/erofs/doc.go Resolved
Comment thread filesystem/erofs/compatibility_test.go Resolved
Replace mkfs-dependent tests and benchmark setup with committed images so
their coverage runs without external tools in CI. Share a reproducible
large fixture covering compact directories, 1 MiB physical clusters,
multi-megabyte decoded extents, and a packed fragment tail.
@deitch
deitch merged commit f6fbf0b into diskfs:master Nov 3, 2026
20 checks passed
@deitch

deitch commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

Looks great. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants