Summary
bubblewrap passes every overlay layer to mount(2) in a single options string. The kernel keeps at most one page of that string and silently drops the rest, so there is a hard limit on how many --overlay-src layers can be stacked. The limit is not documented, and going over it does not produce an error that points at the layer count. It usually produces Overlay directories may not overlap, which is actively wrong: the directories do not overlap, the truncation made them look like they do.
What I hit
Running a container image with 31 overlay layers under bubblewrap 0.12.0:
bwrap: Can't make overlay mount on /... with options lowerdir=...: \
Overlay directories may not overlap
The layers are all distinct sibling directories. Nothing overlaps.
What actually happened is that all the layers live under one layer store, so the paths shared a long prefix and each entry was about 132 bytes:
/oldroot/.../common-storage/overlay/<64 hex chars>/diff
31 of those plus lowerdir= and ,userxattr comes to about 4110 bytes. copy_mount_options() keeps PAGE_SIZE - 1 = 4095, so the 31st path was cut in the middle and became /oldroot/.../common-storage/overlay — the parent of all the other layers. At that point overlayfs is right: one lower layer really is an ancestor of the others. The overlap was manufactured by the truncation.
This cost a while to diagnose, even with the help of Claude Code, because the error message is accurate about what the kernel saw and says nothing about what went wrong.
Current state on main and the just released 0.13.0
Commit c77dd38 ("setup: Use O_PATH fds and disallow symlink targets") replaced the absolute paths with /proc/self/fd/N, which is about 18 bytes per layer instead of 132 (in my case). That raises the ceiling a lot, but it is still a ceiling, and it is still reported as something unrelated to the layer count.
Measured on 8e5e757, Linux 6.12, x86_64, unprivileged:
cd "$(mktemp -d)"
for i in $(seq 1 250); do mkdir -p "l$i"; done
args=$(for i in $(seq 1 250); do printf -- '--overlay-src %s/l%s ' "$PWD" "$i"; done)
bwrap --ro-bind / / $args --ro-overlay /mnt true
- 230 layers: works.
- 250 layers: fails.
The failure is a 4495-byte error message ending in:
bwrap: Can't make overlay mount on (null) with options lowerdir=/proc/self/fd/7:...
...:/proc/self/fd/256,userxattr: Invalid argument
Two separate problems visible in that one line:
- The destination prints as
(null). The loop that collects the lower layers advances op to the last SETUP_OVERLAY_SRC, and those ops carry only a source, so op->dest is NULL by the time the error is formatted.
- The diagnosis is still wrong. Here the truncated
/proc/self/fd/NNN happens to land on a different valid fd rather than on an ancestor directory, so the kernel says Invalid argument instead of complaining about an overlap. Which wrong answer you get depends only on what the cut path happens to name.
So on current main the limit is roughly 230 layers rather than roughly 30, but the shape of the bug is unchanged.
Why this is worth fixing rather than documenting
Container layer stores routinely produce tall stacks, and the failure mode scales badly: the more layers a user has, the more likely the truncated path lands on their shared layer-store parent, which is exactly the case that yields the confusing overlap message.
The layer paths are also not under the user's control in the common case — they come out of whatever produced the image — so "use shorter paths" is not a real workaround. Squashing layers is, but only if you know that is what the error means.
Suggested fix
fsconfig(FSCONFIG_SET_STRING, "lowerdir+", ...) appends one layer per call and accumulates the list inside the kernel, so there is no options string to overflow. Mounting the overlay with fsopen() + one fsconfig("lowerdir+") per layer + fsmount() + move_mount() removes the limit entirely; the only remaining bound is overlayfs's own OVL_MAX_STACK.
I confirmed this works for an ordinary unprivileged user inside unshare(CLONE_NEWUSER | CLONE_NEWNS), which is bubblewrap's model: 300, 400 and 499 layers all mount with every layer visible, and layer 500 fails with EINVAL, which is OVL_MAX_STACK. Unlike open_tree(OPEN_TREE_CLONE), fsopen() creates a new mount instead of cloning an existing one, so none of the cross-namespace restrictions apply.
overlayfs only accepts lowerdir+ from Linux 6.7 on, so older kernels need to keep the mount(2) path. On that path the length should at least be checked up front, so the error names the layer count instead of letting the kernel truncate and overlayfs guess.
This would be a third use of the new mount API after #754 / #756. It is unrelated to #755 beyond the shared theme: that one is about open_tree() for --[ro-]bind-fd and a path-lookup race, this one is a length limit. (I'm going to add a comment to #755 to report on my findings while trying to work that one out.)
I have a patch series implementing the above and will open a PR.
Version / environment
- bubblewrap 0.12.0 for the original report; also reproduced on 8e5e757 (with a larger number of layers).
- Linux 6.12, x86_64.
- Unprivileged (user namespaces), not setuid.
Summary
bubblewrap passes every overlay layer to
mount(2)in a single options string. The kernel keeps at most one page of that string and silently drops the rest, so there is a hard limit on how many--overlay-srclayers can be stacked. The limit is not documented, and going over it does not produce an error that points at the layer count. It usually producesOverlay directories may not overlap, which is actively wrong: the directories do not overlap, the truncation made them look like they do.What I hit
Running a container image with 31 overlay layers under bubblewrap 0.12.0:
The layers are all distinct sibling directories. Nothing overlaps.
What actually happened is that all the layers live under one layer store, so the paths shared a long prefix and each entry was about 132 bytes:
31 of those plus
lowerdir=and,userxattrcomes to about 4110 bytes.copy_mount_options()keepsPAGE_SIZE - 1= 4095, so the 31st path was cut in the middle and became/oldroot/.../common-storage/overlay— the parent of all the other layers. At that point overlayfs is right: one lower layer really is an ancestor of the others. The overlap was manufactured by the truncation.This cost a while to diagnose, even with the help of Claude Code, because the error message is accurate about what the kernel saw and says nothing about what went wrong.
Current state on main and the just released 0.13.0
Commit c77dd38 ("setup: Use O_PATH fds and disallow symlink targets") replaced the absolute paths with
/proc/self/fd/N, which is about 18 bytes per layer instead of 132 (in my case). That raises the ceiling a lot, but it is still a ceiling, and it is still reported as something unrelated to the layer count.Measured on 8e5e757, Linux 6.12, x86_64, unprivileged:
The failure is a 4495-byte error message ending in:
Two separate problems visible in that one line:
(null). The loop that collects the lower layers advancesopto the lastSETUP_OVERLAY_SRC, and those ops carry only a source, soop->destis NULL by the time the error is formatted./proc/self/fd/NNNhappens to land on a different valid fd rather than on an ancestor directory, so the kernel saysInvalid argumentinstead of complaining about an overlap. Which wrong answer you get depends only on what the cut path happens to name.So on current main the limit is roughly 230 layers rather than roughly 30, but the shape of the bug is unchanged.
Why this is worth fixing rather than documenting
Container layer stores routinely produce tall stacks, and the failure mode scales badly: the more layers a user has, the more likely the truncated path lands on their shared layer-store parent, which is exactly the case that yields the confusing overlap message.
The layer paths are also not under the user's control in the common case — they come out of whatever produced the image — so "use shorter paths" is not a real workaround. Squashing layers is, but only if you know that is what the error means.
Suggested fix
fsconfig(FSCONFIG_SET_STRING, "lowerdir+", ...)appends one layer per call and accumulates the list inside the kernel, so there is no options string to overflow. Mounting the overlay withfsopen()+ onefsconfig("lowerdir+")per layer +fsmount()+move_mount()removes the limit entirely; the only remaining bound is overlayfs's ownOVL_MAX_STACK.I confirmed this works for an ordinary unprivileged user inside
unshare(CLONE_NEWUSER | CLONE_NEWNS), which is bubblewrap's model: 300, 400 and 499 layers all mount with every layer visible, and layer 500 fails withEINVAL, which isOVL_MAX_STACK. Unlikeopen_tree(OPEN_TREE_CLONE),fsopen()creates a new mount instead of cloning an existing one, so none of the cross-namespace restrictions apply.overlayfs only accepts
lowerdir+from Linux 6.7 on, so older kernels need to keep themount(2)path. On that path the length should at least be checked up front, so the error names the layer count instead of letting the kernel truncate and overlayfs guess.This would be a third use of the new mount API after #754 / #756. It is unrelated to #755 beyond the shared theme: that one is about
open_tree()for--[ro-]bind-fdand a path-lookup race, this one is a length limit. (I'm going to add a comment to #755 to report on my findings while trying to work that one out.)I have a patch series implementing the above and will open a PR.
Version / environment