feat(volumes): implement E2B volume mounts with Docker named volumes - #4
Closed
Javey wants to merge 7 commits into
Closed
feat(volumes): implement E2B volume mounts with Docker named volumes#4Javey wants to merge 7 commits into
Javey wants to merge 7 commits into
Conversation
Replace absolute ExpiresAt field with computed property derived from LastActiveAt + Timeout. Data-plane requests (any traffic routed through the proxy) now implicitly reset the idle timer via Manager.MarkActivity, eliminating the need for SDK clients to explicitly call /timeout to keep sandboxes alive. Key changes: - Sandbox.ExpiresAt field → ExpiresAt() computed method - Add Sandbox.Timeout and Sandbox.LastActiveAt as sources of truth - Add LabelTimeout Docker label for Rehydrate to recover original TTL - EnforceTimeouts uses idle check (now - LastActiveAt >= Timeout) instead of absolute deadline (now > ExpiresAt) - dispatch.go NewRouter accepts onActivity callback, wired to Manager.MarkActivity in main.go - MarkActivity coalesces writes (default 1s) to avoid lock contention - Config: EDVABE_KEEPALIVE_ENABLED (default on), EDVABE_KEEPALIVE_COALESCE - API responses and dashboard include lastActiveAt field Design inspired by CubeSandbox's traffic-driven keepalive model.
KeepaliveEnabled was a bool with broken default handling — both branches of the if set true, making it impossible to disable keepalive. Changed to *bool: nil = default on, &true = explicit on, &false = off.
Drop *bool three-state approach. main.go is the only Options constructor and always passes a concrete bool. The original bug was just redundant if-branches — a direct assignment fixes it.
Pre-flight ContainerInspect before pause/unpause: Pause returns early if already paused, Unpause returns early if already running. Robust against Docker API message changes across versions. Fixes the Manager TOCTOU race (Resume/Connect check State==Paused under lock, release, then call unpause - a concurrent resume wins the race) and enables recovery from external docker commands.
Data-plane traffic (any request routed through the proxy) now auto-resumes a paused sandbox before forwarding. This mirrors CubeSandbox sandbox_state.lua gate: a browser refresh or SDK request to a paused sandbox transparently unpauses it instead of hanging or erroring. NewProxy gains a third AutoResumer interface param (satisfied by Manager.Connect, which unpauses + resets the idle clock). Running sandboxes skip the check entirely. Paused sandboxes get Connect(id, sb.Timeout) — preserving the original TTL while granting a fresh idle lease. Two new tests: TestAutoResumePausedSandbox verifies the resume+forward path; TestProxyNoResumeForRunningSandbox verifies no spurious Connect for running sandboxes.
Implements the full volume mounts plan from docs/08-volume-mounts-plan.md:
Volume CRUD:
- POST /volumes creates a Docker named volume (edvabe-vol-<volID>)
with edvabe labels, chowns root to UID 1001 via busybox helper
- GET /volumes lists managed volumes (filtered by label)
- GET /volumes/{id} returns volumeID, name, token
- DELETE /volumes/{id} removes the volume (409 if in use)
- In-memory name→ID index rebuilt from Docker labels on startup
Sandbox creation:
- volumeMounts field typed as []sandboxVolumeMount (name + path)
- Validation: name must exist, path absolute, no /, no dupes, max 16
- Resolved mounts passed through CreateOptions → CreateRequest.Mounts
- Docker runtime converts to mount.TypeVolume entries in HostConfig
Persistence:
- Logical {name, path} stored in edvabe.sandbox.volume.mounts label
- Rehydrate restores VolumeMounts from the label
- Docker container mount config survives pause/stop/start/restart
API responses:
- sandboxDetailResponse includes volumeMounts [{name, path}]
- create response does not include it (matches upstream SDK)
Runtime interface:
- Replaced BindMounts map[string]string with typed []Mount
- Added VolumeCreate/VolumeList/VolumeInspect/VolumeRemove methods
- noop runtime has in-memory volume store for tests
Envd boundary:
- agent.VolumeMount struct corrected to match envd NFS shape
(nfs_target + path) so it cannot be accidentally used for local
Docker volumes
- InitAgent continues to receive empty volumeMounts — Docker handles
mounting at container creation, envd NFS code never fires
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the full volume mounts plan from
docs/08-volume-mounts-plan.md.Changes
Volume CRUD (runtime-backed, replaces in-memory stub)
POST /volumes→ creates Docker named volume (edvabe-vol-<volID>) with edvabe labels, chowns root to UID 1001 via busybox helper containerGET /volumes→ lists managed volumes (filtered by Docker label)GET /volumes/{id}→ returnsvolumeID,name,tokenDELETE /volumes/{id}→ removes volume (409 if in use)Sandbox creation with volumeMounts
volumeMountsfield typed as[]sandboxVolumeMount({name, path}) — replaces[]json.RawMessage/, no duplicate paths, max 16 mountsCreateOptions→CreateRequest.Mounts→ DockerHostConfig.Mountsasmount.TypeVolumePersistence & rehydration
{name, path}stored inedvabe.sandbox.volume.mountsDocker label as JSONRehydraterestoresVolumeMountsfrom the labelAPI responses
sandboxDetailResponseincludesvolumeMounts: [{name, path}]Runtime interface changes
BindMounts map[string]stringwith typed[]Mount(ordered, supports bind + volume)VolumeCreate/VolumeList/VolumeInspect/VolumeRemovetoruntime.RuntimeEnvd boundary
agent.VolumeMountstruct corrected to match envd NFS shape (nfs_target+pathJSON tags) — prevents accidental use for local Docker volumesInitAgentcontinues to receive emptyvolumeMounts— Docker handles mounting at container creationTest plan
go vet ./...passesgo test ./...passesTestVolumesLifecycle— create, list, get, delete volumeTestCreateSandboxWithVolumeMounts— create sandbox with volume, verify get response echoes mountsTestCreateSandboxRejectsUnknownVolume— 400 for unknown volume nameTestCreateSandboxRejectsInvalidVolumeMounts— 400 for empty name, relative path, root path, duplicate pathTestVolumeDuplicateName— 409 for duplicate volume nameTestCreateWithVolumeMounts— manager creates sandbox, runtime receives correct mount entriesTestRehydrateRestoresVolumeMounts— volume mounts survive manager restart via Docker labels