Skip to content
Ephemeral-AI-LabPublic

About

Ephemeral Workspaces, durable shared history. LayerFS gives every agent a complete, writable filesystem Workspace over shared committed history, without copying the base. Agents work in parallel, commit what changed as deduplicated checkpoints, then branch, rewind or merge. Try it with LayerFS × DSH.

Topics

Resources

Stars

1.3k stars

Watchers

1 watching

Forks

Latest commit

 

History

4,033 Commits

Folders and files

Repository files navigation

LayerFS

LayerFS

Ephemeral Workspaces. Durable Shared History.

LayerFS is the substrate under your agents. Each one mounts a complete, writable Workspace over shared committed history, does its work, commits what it changed, and goes away.

Get started · Website · Docs · Benchmarks · Video · LayerFS × DSH · Community

MIT license 1.0.0 Linux, and macOS through DSH Discord

layerfs-v1.0.0-release-720p.mp4

Status: LayerFS 1.0.0, the first official release after the 0.1.x Developer Previews, released as source. The results below ran on main at e5c7aaae05. The Store runs in its Disposable profile only, with no crash or power-loss durability: use LayerFS for agent work and its history, not as the only copy of data you cannot rebuild.

Try it with LayerFS × DSH

LayerFS × DSH: the LayerFS logo and the DeepSeek Harness logo

LayerFS runs today through LayerFS × DSH, the dsh-layerfs plugin for DeepSeek Harness. Agents in a dsh session fork Branches, test them in Sandboxes, merge and publish, and the Work and History tabs show every agent's Branch, live.

Try LayerFS × DSH: install guide

Core ideas

fs calls go straight into the Store as Commits; each Bash call mounts a Workspace, runs, commits and unmounts
01 · Ephemeral Computing
Compute exists only while a command runs.
Four agents, each on its own computer, read and commit to one shared Store
02 · Decentralized Agents
Agents on different computers share one serverless Store.
Main with two Branches, three agents on each; a commit to a file that already changed is refused with a Conflict
03 · Multiagent Coordination
One project, many Branches. One Branch, many agents.
Agents fork instantly into new Branches, recursively; each Branch is scored and the best one is kept
04 · Recursive Filesystem
Fork recursively any Commit, at any depth.
Four agents commit the same packages; the first stores every chunk and each later Commit adds only what is new
05 · Ultra Storage Optimization
157 snapshots of a real project in 100 MB, 19.9× smaller than Btrfs + zstd.
Without ignored files the tests fail until packages are installed; with LayerFS the mount already has them
06 · Instant Bootstrap
Dependencies come with every mount. No time lost reinstalling.

And underneath them

Supporting idea In one line
07 · Auto Checkpoints Every step an agent takes is a checkpoint to return to.
08 · Append-only Objects Saved objects are never rewritten. The Store only grows.
09 · Conflict Rules (.lww) Name the files that should never conflict. The newest edit wins.
10 · Big File, Small Edit An edit costs the size of the edit, not the size of the file.

See it run

The LayerFS demo: agents on several Branches run tool calls in Sandboxes and fs, each call becomes a Commit, and the history tree grows

Agents fork Branches, work in Workspaces inside Sandboxes, commit, merge and publish to main. The history grows as they go.

Results

Every suite ran once on main at e5c7aaae05 through the JavaScript SDK. Each row links to its method, environment and receipts.

Suite What it measures LayerFS 1.0.0
fs-bench Cloudflare Computer's own twelve-scenario filesystem benchmark, unchanged, same image 5.20 s vs 8.88 s, 1.71× faster; faster in 10 of 12 scenarios
dsh-dedup A real project's history, 157 snapshots (4.94 GB of trees), kept as mountable Commits 100.1 MB vs 1,994.6 MB, 19.9× smaller than Btrfs + zstd; 156/156 read back
pjdfstest POSIX conformance, 8,798 assertions, against kernel overlayfs 0 unexpected failures; keeps 76% of overlayfs's passes, every other failure a deliberate format limit
fsx Random reads, writes, mapped I/O and truncates, every byte checked 14/14 pass, 3.06 M ops, 0 miscompares
fsstress Four concurrent processes, then Commit, remount and compare 3,755/3,755 entries identical; 5.21 GB captured by one Commit in 4.9 s
smallfile 100,000 files of 4 KiB, four threads, against libfuse passthrough Faster in 9 of 9 operations, 1.27×–10.71×
mdtest Metadata create, rename and remove, against libfuse passthrough 0.86–1.28× with 1 process, 1.21–4.22× with 4
fio Data I/O bandwidth, against libfuse passthrough Random read/write mixes 1.49–2.23×; slower on sequential write (0.31×), sequential read (0.81×) and random write (0.40×)

One sample per case, seed 1, no best-of. Host: arm64 Mac, Docker Desktop linux/arm64, 8 CPUs, 8.3 GB; host load is recorded per suite. These are release measurements, not a guarantee for your workload. Earlier fs-bench-pro results: v0.1.4 · v0.1.5 · v0.1.6.

How it works

agent tool call
 ├─ file tool (read, write, edit, grep …) ── SDK, in process ──────► Store
 └─ Bash ── mount ─► exec ─► commit ─► unmount                       ▲
             └─ Workspace: FUSE view in a Sandbox's daemon ───────────┘
                (committed root + its own overlay)
  • Store. One SQLite file holds every Branch, Commit and content object. Each daemon and SDK process opens it directly; there is no database server.
  • Branch, Commit, Layer. A Branch is a writable line of Commits. Publishing a Branch to main adds a Layer. Content is chunked and stored once.
  • Workspace. A mount binds the whole committed tree without copying it. Writes go to the Workspace's own overlay until a Commit saves them. Unmount deletes the overlay, never history.
  • Conflict. Two Commits that change the same file are refused with Conflict, unless a .lww rule says the newest edit wins.

Learn more

Docs LayerFS × DSH guide · layerfs.ai/docs · Core ideas · JavaScript SDK guide
Concepts Core concepts · Documentation index
Benchmarks layerfs.ai/benchmarks
Release 1.0.0 plan · Previous: 0.1.7 release record and its manual

Community

Questions, ideas and show-and-tell are welcome in either place.

Discord

Join the LayerFS Discord

discord.gg/8mmAMCDc2
WeChat · LayerFS的好朋友

QR code for the LayerFS WeChat group
Scan in WeChat to join the group.

License

LayerFS is licensed under the MIT License.

About

Ephemeral Workspaces, durable shared history. LayerFS gives every agent a complete, writable filesystem Workspace over shared committed history, without copying the base. Agents work in parallel, commit what changed as deduplicated checkpoints, then branch, rewind or merge. Try it with LayerFS × DSH.

Topics

Resources

Stars

1.3k stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages