Skip to content

Free workers when their member model host leaves - #427

Open
clocksmith wants to merge 1 commit into
Nehanth:mainfrom
clocksmith:fix/member-host-departure-354
Open

clocksmith wants to merge 1 commit into
Nehanth:mainfrom
clocksmith:fix/member-host-departure-354

Conversation

@clocksmith

@clocksmith clocksmith commented Oct 30, 2026 •

Copy link
Copy Markdown

Fixes #354.

Behavior

When a member model host closes or reloads, surviving workers now free their layers and return to Start. Abandoned loads stop, including during pacing waits; late async completion cannot restore stale readiness. Creator recovery and transient link replacement are preserved. No wire-format or kernel changes.

Regression evidence

Compared base 6b4a24f with this PR's 6988cd6 using these browser regressions.

Scenario Trigger Base Fix
hostgone Loaded member host closes Both survivors retain stale engines; cleanup fails Cleanup, restart and answer pass
hostreload Member host reloads with a new peer ID Both survivors retain stale engines; cleanup fails Cleanup, restart and answer pass
hostgone-loading Host leaves before a held worker load completes Late completion retains abandoned layers/readiness; cleanup fails Late completion is cancelled; cleanup, restart and answer pass

The three scenarios contain 18 checks. Base fails all six survivor-cleanup assertions; fix passes 18/18. The 18 focused checks are included in the 101-check full suite; repeated environment runs below are the same checks, not additional unique tests.

Hardware and browser coverage

Hardware specs were verified with read-only inventory after the tests; browser versions and WebGPU backends were recorded during the runs.

System Operating system CPU GPU / backend System RAM and GPU memory
Apple M3 system (MacBook Air) macOS 26.6.2 Apple M3, 8 CPU cores Integrated Apple M3 10-core GPU, Metal 24 GiB unified memory shared by CPU and GPU; not dedicated VRAM. Separate WebGPU memory budget not recorded.
Core i3-1115G4 system Ubuntu 24.04.3 LTS, Linux 6.8.0-90-generic, x86_64 Intel Core i3-1115G4 @ 3.00 GHz, 2 cores / 4 threads Integrated Intel UHD Graphics G4 / Tiger Lake-LP GT2 (WebGPU: Gen-12LP), Vulkan 32 GiB system RAM; integrated GPU uses shared system memory. Dedicated VRAM capacity and shared GPU memory budget were not reported by the available inventory.
Setup Browser versions Result
Apple M3 system, separate browser contexts Chromium for Testing 143.0.7499.4 Full room-state suite 101/101; focused subset 18/18
Core i3-1115G4 system, separate browser contexts Installed Chrome 136.0.7103.59 Focused 18/18; matching base cleanup failures reproduced
Physical Apple M3 ↔ Core i3-1115G4 room, native GPUs Apple M3 system: Chromium 143.0.7499.4; Core i3 system: Chrome 136.0.7103.59 Base fails all six cleanup assertions; fix 18/18
Physical Apple M3 ↔ Core i3-1115G4 room, installed browsers Apple M3 system: Chrome 156.0.8078.74; Core i3 system: Chrome 136.0.7103.59 Fix 18/18

Physical-room topology: member model host on the Core i3-1115G4 system; creator and worker on the Apple M3 system—three browser contexts across two physical computers, communicating over real WebRTC UDP host-candidate pairs. A physical-room repeat using Apple Metal on the M3 GPU and SwiftShader on the Core i3 system also passed 18/18; SwiftShader independently reproduced the bug on the Apple M3 system. Test-only context-routing and browser launch overrides are excluded from this PR.

Additional test outcomes

  • Actual Safari: the same hostgone, hostreload, and hostgone-loading scenarios pass 18/18 on the fix; base fails all six cleanup assertions.
  • Deno (applicable checks): focused checks pass 17/17 on the fix versus 16/17 on base, where the cancellation regression fails. GPU-free shared-runtime checks pass on both.
  • Windows Chrome: validation is blocked by test-harness setup, remains unverified, and is not counted as a pass.

Other checks and limitations

  • Focused lifecycle/authorization units: 17/17. Serialized room-node/OpenClaw packages: 212 passed, seven skipped. Headless runtime, syntax and patch-application checks passed.
  • Browser runs use a tiny synthetic GGUF substituted for the 27B selector, not full-size 27B/35B weights.
  • Full local units under Deno 2.7.5: 1,397 passed, 14 timer/async-leak failures, five ignored, plus an fs import error. Those failure cases reproduce on base; CI pins Deno 2.9.5.
  • Preview: 59/60 on both base and fix, with the same two-second visible-preview recovery timeout. Complete unit/preview suites are not claimed green.
  • Hosted CI is currently action_required before jobs run, awaiting maintainer action. Vercel also requires deployment authorization.

AI-assisted implementation; source and regression results were inspected locally.

Cancel abandoned shard loads, release worker resources and return survivors to Start. Guard late load continuations and pacing waits so a new model can start without reloading tabs.

Add browser regressions for host close, reload and late completion, plus cancellation and creator-recovery unit tests.
@clocksmith
clocksmith requested a review from Nehanth as a code owner October 30, 2026 14:48
@vercel

vercel Bot commented Oct 30, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the nehanth's projects Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Room: devices holding layers stay "online" after a member model host leaves

1 participant