Skip to content

fix(rust): avoid copy-up on read-only open - #343

Open
5214jerry wants to merge 2 commits into
tursodatabase:mainfrom
5214jerry:fix/rust-read-only-open
Open

5214jerry wants to merge 2 commits into
tursodatabase:mainfrom
5214jerry:fix/rust-read-only-open

Conversation

@5214jerry

Copy link
Copy Markdown

Summary

  • Avoid copying base-layer files into the SQLite delta when they are opened
    read-only.
  • Preserve copy-up behavior for O_WRONLY and O_RDWR opens.
  • Add Rust SDK regression tests for both read-only pass-through and read-write
    copy-up.

Problem

OverlayFS::open() previously ignored its flags argument. Opening any inode
from the base layer called copy_up_and_update_mapping(), including
O_RDONLY opens made by tools such as grep, cat, and repository scanners.

For regular files, copy-up reads the entire base file and writes it into the
SQLite delta. A read-only repository scan could therefore materialize thousands
of unchanged files even when the agent edited only one or two files.

Change

The base-layer branch now checks the requested access mode:

  • O_WRONLY or O_RDWR: copy up and open the delta file as before.
  • Otherwise: open the base inode directly with O_RDONLY and return it without
    creating a delta file or origin mapping.

O_RDONLY is zero, so the implementation detects write access through
O_WRONLY and O_RDWR rather than testing O_RDONLY directly.

The pass-through path intentionally supplies O_RDONLY instead of forwarding
all caller flags. On Linux, HostFS reopens an inode through
/proc/self/fd/<fd>, which is a symlink; forwarding O_NOFOLLOW can otherwise
cause ELOOP for read-oriented tools such as recursive copy operations.

Tests

Added two Rust SDK regression tests:

  • test_overlay_read_only_open_does_not_copy_up
    • verifies that the base contents remain readable;
    • verifies that no delta file is created;
    • verifies that no origin mapping is created.
  • test_overlay_read_write_open_copies_up
    • verifies that a delta file and origin mapping are created;
    • verifies that writes are visible through the overlay;
    • verifies that the base file remains unchanged.

Targeted test result:

running 2 tests
test filesystem::overlayfs::tests::test_overlay_read_only_open_does_not_copy_up ... ok
test filesystem::overlayfs::tests::test_overlay_read_write_open_copies_up ... ok

test result: ok. 2 passed; 0 failed

Measured impact

The released v0.6.4 binary and a binary containing this change were compared
on five SWE-bench instances. “From base” counts file content associated with an
fs_origin row and excludes files newly created by the agent.

Instance v0.6.4 from base Patched from base Delta footprint, before → after
django-14999 96.92 MB / 2,252 files 0.03 MB / 2 files 261.9 → 20.6 MB
django-11179 94.01 MB / 2,132 files 0.00 MB / 1 file 257.6 → 19.4 MB
matplotlib-23913 101.22 MB / 1,122 files 0.05 MB / 2 files 239.0 → 18.9 MB
sklearn-13497 50.62 MB / 751 files 0.02 MB / 3 files 134.7 → 13.3 MB
sympy-16503 84.73 MB / 1,238 files 0.09 MB / 2 files 222.5 → 31.8 MB

The remaining patched footprint is primarily agent-created cache data rather
than read-induced repository copy-up.

Additional validation

The patched binary was validated on a 58 MB repository containing 1,235 files:

  • a full-tree read sweep did not materialize repository contents in the delta;
  • an MD5 comparison through the mount reported zero mismatches;
  • writing an existing base file still copied it up and left the base unchanged;
  • recursive read/copy flows continued to work with the O_RDONLY base open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant