[FeathBow]

Summary

Part of #162, the second half of step 5: retire MAX_TREE_DIRECTORIES = 1023, and the directory bound with it.

The walk held a tree to 1,023 directories, root included, because recovery refused more than 1,024 live seals and one of them goes to the source parent. The first half of this step (#227) took that refusal out of recovery. Nothing else needs a directory bound of its own. A directory is an entry, and max_entries = 100_000 already bounds entries in every walk and every manifest decoder. What a directory adds is a seal and its inverse, and that is what WAL admission (#220) measures against the room the WAL has. File descriptors do not grow with directories: recovery reopens one chain at a time, and the 240-seal descriptor test still passes under 128. Time grew with their square in two places, and both are removed here. So the bound goes rather than rising: max_directories, HeldTreeLimit::Directories and InvalidDirectoryLimit are removed, and the walks count directories only for the tree's shape.

The count #217 added past the bound goes with it. It let a refusal say how many directories a tree has, and no tree is refused for that any more, so HeldTreeError::DirectoryCount, the assessment's count-past mode, HeldTreeLimitExceeded, and the preview's limit and observed fields are removed. clean -n --json no longer reports directory_limit_exceeded or invalid_directory_limit; a tree too large is refused for its entries, or as wal_full when the WAL has no room for it. The purge's walk over directory attributes is held to max_entries now, and the manifest decoder no longer counts directories.

One is in checking a staged tree against its manifest, which a staging and a purge each do once and an undo twice. The check looks up each directory's sealed and current modes in three passes over the manifest, and each lookup scanned every seal plan. It now asks an index of the plans by path, built once per check. The other is in the undo: after restoring the modes, it rewrote the mode recorded for each directory on every plan's chain, searching the plan list for each one. Nothing reads those modes once they are restored, so the rewrite is deleted rather than indexed. Timed alone in a release build on synthetic plans, one pass of the old lookups took 14.0 s at 7,705 directories and 81.2 s at 20,000. One pass through the index takes 0.13 s at 100,000.

docs/safety.md drops the 1,023 from the inventory bounds and the paragraph about counting past it, and says next to the downgrade note that a binary which still bounds directories at 1,023 cannot recover, undo or purge a tree staged above that.

Measured end to end at the sizes #162 reported, on macOS APFS. Every step ran in a fresh process under ulimit -n 128; peak RSS is from /usr/bin/time -l.

	npm-sized: 3,825 directories, 16,698 entries	Cargo-sized: 7,705 directories, 52,207 entries
clean -n	1.9 s, 26.3 MiB	6.4 s, 72.3 MiB
clean, staging it	49.4 s, 50.8 MiB	127.1 s, 108.7 MiB
WAL held once staged, committed + reserved	0.74 + 0.82 MiB	1.51 + 1.65 MiB
undo	46.2 s, 45.4 MiB	111.1 s, 68.1 MiB
the next session after a staging killed halfway, rolling it back	18.0 s, 24.8 MiB	30.5 s, 38.1 MiB
clean --purge, staging and purging it	61.6 s, 55.0 MiB	160.6 s, 85.5 MiB

The trees are synthetic, built breadth-first with eight subdirectories to a directory and 64-byte files spread over them, to the counts #162 reported. A staging is killed once it has written half the WAL bytes a whole one writes. The binary is a debug build, because only the test build honors the integration anchor that keeps a run off the real account.

Measured the same way before the index, staging, undoing, and staging and purging the Cargo-sized tree took 150.5, 170.2 and 233.6 s, and the npm-sized one 59.2, 63.2 and 77.1 s. What the index saves an undo grows from 17.0 s to 59.1 s when the tree doubles. Memory and descriptors stay small. The time left is about 16.5 ms a directory to stage.

Validation

cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features --locked (also in a separate target directory), RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --locked and cargo test --workspace --locked are clean, the suite at 1816 passed / 0 failed / 1 ignored.

Each test below builds a tree of 1,100 directories, 77 past the old ceiling:

• old_ceiling_admitted: the v2 proof walk, the assessment and the schema-1 walk each admit it and count all 1,100 directories. With max_entries at 1,050, all three refuse the same tree for its entries.
• public_size_hardlinks: the public assessment admits it and reports 1,100.
• production_sealed_staging_cli_clean_undo_and_direct_purge (CLI), raised from the 1,023-directory boundary it used to pin. Each step is a fresh process under a descriptor ceiling of 64: a preview assesses the tree, clean stages it, undo restores every directory's mode and every file's contents, a purge preview sizes it in the WAL within the room available, and the purge deletes it.
• wide_tree_cleans (TUI): the review starts it in the plan and stages it on confirmation.
• wide_recovery: with 1,101 live seals, a process killed after every directory is sealed is rolled back at the next open, and an undo stopped at its first inverse is finished there.

modes_by_path checks the index on its own. A directory's modes come back for its path, as restored or as sealed. A path two plans name, a path no plan names, and every path once a plan lies outside the destination root are refused, as the scan refused them.

The tests of the bound itself are gone: the explicit 1,024-directory request, the 1,023/1,024 boundary, the four tests of counting past it, the decoder's directory limit, and the preview's limit/observed fields.

Seven mutations, all caught. Putting a 1,023-directory ceiling back into the walk turns all five tests above red. Not counting directories, holding the directory-attribute walk to 1,023, and refusing more than 1,024 live seals in recovery again are each caught by the test for that path. Letting the index answer for a path two plans name, or ignore a plan outside the destination root, is caught by modes_by_path alone; letting it ignore restored modes is caught there and by 13 staging tests. Deleting the undo's rewrite of chain modes leaves every test green, which is why it is deleted.

The three end-to-end tests take 11 to 33 seconds each on macOS APFS and use the CPU for under a quarter of that. The rest is waiting on the WAL sync before and after each seal, about four per directory over a stage, an undo and a purge, which a release build would not shorten. Batching those syncs is a separate change.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: e7c8a692-e6c1-4a99-ae15-a969c58181d7

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}