Skip to content

Let a nibblet run in a virtual machine - #14

Open
IAmJSD wants to merge 60 commits into
mainfrom
virtual-machines
Open

IAmJSD wants to merge 60 commits into
mainfrom
virtual-machines

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

A nibblet can now run in a virtual machine of its own instead of a container: Linux on any host with a hypervisor, Windows on Hyper-V (and KVM on x86), and macOS on Apple silicon. Containers stay the default and the recommended way, and Docker is still needed for the stack.

How it fits together

  • Machines are plugins. @nibblet/machine is the base: MachinePlugin, the contract the conductor uses for a nibblet's machine, and a registry that routes each nibblet to its own. @nibblet/machine-docker is today's container code moved behind it, with no change in behaviour. A nibblet records machine: { backend, guest } when it's made.
  • Hypervisors run in the host bridge, since the server is in a container: machine-apple (a small Swift helper, nibblet-vm, on Virtualization.framework), machine-hyperv (PowerShell) and machine-kvm (QEMU). The server's VirtualMachines plugin speaks to them over the bridge's new /machines API.
  • A daemon on a virtual serial port. Each VM's guest agent runs as root from boot on a port of its own, which the bridge relays to the server. Over it the server does what docker exec does for a container. The runner's own connections to the server and credential proxy go out over the same port, so the guest needs no route to the host, and nothing in it holds a key.
  • Images. Linux guests are the runner image itself, with a kernel and systemd added, laid out as an EFI disk by Docker on the host. Windows is installed unattended from your own ISO. macOS is installed from Apple's restore image and finished offline, which takes one administrator's password.
  • The entry is a platform plugin too: Linux's (unchanged), macOS's (dscl users, pf) and Windows' (the guard as SYSTEM, the agent as a hidden local user, ACLs, Windows Firewall).
  • In the app: a "Runs in" picker when making a nibblet (shown only where there's a hypervisor), its settings leave out what its machine can't do, and Settings has a section for the images. install.sh offers QEMU where there's /dev/kvm, off unless asked for.

See the new "Machines" section of ARCHITECTURE.md.

What works, and what's tested

An integration suite (test/integration) runs nibblets through a real server: it brings up a stack of its own beside anything else on the engine, with a fake Anthropic API where the credential proxy would be, so the real Claude Code harness runs with no credentials. Each run makes a nibblet in one machine and guest, has it answer, opens its browser (where the guest has one), snapshots and restores it, stops and wakes it, has it spawn a sub-agent, and deletes it, checking nothing's left on the host.

Machine Guest State
Docker Linux Integration suite passes in CI.
Podman Linux Integration suite passes in CI. It found that snapshots never restored on Podman, now fixed.
KVM Linux Integration suite passes in CI, on real /dev/kvm.
Apple Virtualization (M4, macOS 15.6) Linux Booted by hand: entry, guard, browser view, home archive, restart keeping its home, clean stop. Suite not run here yet.
KVM Windows Integration suite passes in CI (the Machines workflow, from a Windows 11 ISO in the WINDOWS_ISO_URL secret): unattended install in about 30 minutes, then the whole suite.
Hyper-V Linux, Windows Integration suite passes in CI for both, on windows-2025, which runs Hyper-V (and WSL 2) nested: a NAT switch with DHCP for the guests, and Docker in WSL for the stack.
Apple Virtualization macOS Suite written, to run on a Mac (a 15 GB restore image, and an administrator's password). Not run.
  • Windows and macOS guests install the runner from GitHub, so the server can now name the git ref they install (NIBBLET_GUEST_REF, the release's tag by default); the suite uses the commit under test.
  • Getting Windows through CI found and fixed real bugs: the first-run screens weren't answered, Enter for the DVD's boot prompt could cancel setup, the runner couldn't read packages hard-linked from SYSTEM's pnpm store, the agent's folders were granted to the computer rather than its user (both named nibblet), Windows' disk overlay was cut to 32 GB, and the COM relay dropped every connection at a line error. Windows' install now says each step on COM1, KVM screenshots it, and it fails with why rather than hanging.
  • Snapshots never restored on Podman (dockerode's attach body reached tar's stdin); fixed, and the suite covers it.
  • An in-process test runs the whole chain (server plugin, bridge API and relay, the guest daemon, a fixture entry) without a VM.
  • Unit tests cover the ISO writer, autounattend.xml, the Windows and macOS firewall rules and ACLs, KVM's devices, and the host's image jobs.
  • 434 tests pass, plus prettier and shellcheck.
  • So a test stack can share an engine, each server labels its machines with its stack (NIBBLET_STACK) and only reconciles its own.

Known limits

  • Linux guests only have the browser, the desktop and Tailscale for now. No VM shares the host's SSH agent.
  • A VM's CPU and memory apply when it next starts.
  • Windows on KVM talks to the host over a COM port, which works but is slow for big homes; guestfwd would be the faster follow-up. (Hyper-V guests use Hyper-V sockets.)
  • A Windows nibblet's product key can be given when it's made, not yet changed afterwards.
  • Windows and macOS guests install the runner from the release the server runs (v<version>), so they need a release with this in it.
  • On Windows, stopping a nibblet ends it at once: there's no SIGTERM for console processes.
  • Booting Linux on M4 Macs needed arm64.nosme and no memory balloon.

Since: Hyper-V in CI, and Windows product keys

  • Hyper-V sockets. Hyper-V guests reach the host over a Hyper-V socket rather than a COM port, which lost what the host sent a Linux guest. A relay in the guest (Python for Linux, nibblet-hvsock.exe for Windows) takes only the host's connections, so nothing in the VM can pose as it; the bridge connects through a helper it builds with Windows' own csc.
  • Windows Server and Docker without Docker Desktop. Hyper-V goes out through a named switch where there's no Default Switch (--hyperv-switch), and VHDX Linux images are copied out of their build container rather than written to a mount, so Docker in WSL works.
  • Windows product keys. An optional key when making a Windows nibblet, installed and activated in its VM whenever it boots unactivated, said in its thread, and never sent back unmasked or put anywhere the runner can read.
  • Bugs this found: a string sent to a guest was framed as no bytes and crashed the server; Hyper-V's capacity check asked for a switch with no name; a Linux guest's COM port ran at 9600 baud.

🤖 Generated with Claude Code

IAmJSD and others added 30 commits October 17, 2026 14:33
Where a nibblet runs is now a plugin: @nibblet/machine is the base, with
the contract the conductor uses and a registry that loads plugins and
routes each nibblet to its own, and @nibblet/machine-docker is today's
container code moved behind it. Nibblets record their machine (backend
and guest), every existing one a Docker container running Linux, and the
browser, Tailscale and the SSH agent are only offered where the machine
can have them.

Nothing changes for containers. This is the ground for virtual machines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A nibblet's machine can now be a VM on the host, made by a hypervisor
plugin in the host bridge: Apple Virtualization (a small Swift helper,
nibblet-vm), KVM through QEMU, and Hyper-V through PowerShell. The
server's VirtualMachines plugin fulfils the same contract as Docker for
all of them, through the bridge's /machines API.

Inside each VM a guest agent, run as root from boot, sits on a virtual
serial port of its own and does what docker exec does for a container:
runs the entry's commands (the nibblet, sub-agents, home archives),
opens ports for the browser's view, and carries the runner's own
connections to the server and the credential proxy out to the bridge,
which makes them on the host. The port is the only way in or out, so the
guest needs no route to the host and nothing in it holds a key.

Linux guests run the runner image itself, with a kernel and systemd
added, laid out as an EFI disk by Docker on the host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Booting real guests with Apple Virtualization on an M4 found what a
container never had to: Tailscale's own daemon enabled under systemd,
the memory balloon (with it, the guest's pages are taken back under it),
the EFI partition's automount, and arm64's SME. The agent's port is now
a second virtio serial console (the multiport console's named ports
carried nothing), set raw while it's open, and the helper keeps its
pipes alive. Making the image keeps one copy of the guest's filesystem
rather than four, and checks there's room first.

A Linux nibblet in a VM boots to its entry in about four seconds, with
the browser, snapshots and sub-agents as in a container.

In the app, a new nibblet can be put in a VM where the server's host has
a hypervisor (containers stay the recommended way), its settings say
where it runs and leave out what its machine can't do, and Settings has
the images. The desktop app and the CLI bridge load the hypervisor for
their OS, and the credential proxy is on the host's loopback for the
bridge to reach.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The entry is now a platform plugin like the rest: Linux's (iptables,
setpriv, Tailscale) as before, macOS's (dscl users, pf in an anchor of
its own) and Windows' (the guard as SYSTEM, the agent as a hidden local
user started by a small helper that ends its whole tree, ACLs, and
Windows Firewall). The runner's tasks and terminals work on all three:
PowerShell and taskkill on Windows, node-pty where there's no Python.

Windows guests (Hyper-V, and KVM on x86) are made from the user's own
installer: an answers disc made here (a small ISO writer) installs
Windows unattended, and a bootstrap run at every startup installs Node
and the release's runner, builds the helpers, and runs the agent on
COM2. macOS guests (Apple silicon) install the newest macOS Apple offers,
then get the agent offline, with one administrator's password; the
host's own VM helper relays their vsock port.

KVM runs Linux guests from the same disk as Apple's, and install.sh
offers QEMU where there's /dev/kvm, off unless asked for: containers
stay the recommended way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bridge asks each hypervisor what it can do at most every half minute,
and Hyper-V reads every nibblet VM's state in one go: its cmdlets are
PowerShell, which is slow to start, and the server asks often.

ARCHITECTURE.md describes machines: the plugins, how a VM's guest agent
is reached, and how each guest's image is made.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Booting a Linux guest with QEMU found that its network card wanted iPXE's
ROM, which distributions package apart; nothing boots from the network,
so neither card has one. The plugin can also run on TCG, QEMU's own CPU,
for testing where there's no KVM: a Linux guest booted that way, under
Docker Desktop on a Mac, all the way to its entry, its gate, its home
archive and a clean power-off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A VM's guard and agent get no-new-privileges and an empty bounding
  set, as Docker gives a container's; its home disk is frozen while it's
  archived, as a container is paused, and always thawed after.
- An old exit in a hello no longer settles a run that's running again,
  and jobs' names are unique to their link.
- A port that starts afresh cuts off its channels: a job whose output
  was lost fails, rather than a short archive passing as a snapshot.
- An entry that ends by itself takes its VM down with it, and a new link
  notices an entry that went while nobody was linked.
- A VM booted for a job alone isn't taken for a running nibblet.
- The bridge holds the server back when a guest's port is full, Apple's
  stop only signals its own helper, a hypervisor that can't say how a
  VM is no longer means it's off, and VMs' sockets are in a folder only
  the bridge's user can open.
- Tunnels are shared by requests that ask at once, links time out, the
  losing side of a job's race can't crash the server, and closed
  connections finish what they were sending.
- Linux images' roots are full size, rather than grown online at boot,
  and stopping a VM tries its power button before its plug.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An integration suite that brings up a whole stack of its own beside
whatever else is on the engine, with a fake Anthropic API where the
credential proxy would be, so the real Claude Code harness runs with no
credentials. It makes a nibblet in the machine NIBBLET_TEST_MACHINE
names (docker, apple, kvm or hyperv), has it answer, opens its browser,
snapshots and restores it, stops and wakes it, has it spawn a sub-agent,
and deletes it.

So a test stack can share an engine, the server labels its machines
with its stack (NIBBLET_STACK) and only reconciles its own, and image
pruning and harness updates can be turned off.

CI runs it on Docker, Podman and KVM; Apple's and Hyper-V's run locally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude Code sends its own context after the prompt, in a message of its
own, so the fake now reads the whole of the user's turn. It also keeps
what each call offered and answered, for failures to explain themselves.

Podman's socket is in a folder only root can open. A server with no
bridge, or none of a hypervisor on its host, has none of its VMs, rather
than failing to list them on every reconcile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Attaching to a container sent dockerode's options as the request's body,
which Docker ignores but Podman takes as the start of stdin, so tar got
70 bytes of JSON before the archive and every restore failed. The home
helper is now an idle container that runs its command as an exec, whose
stdin is only what's piped, on either engine.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The integration suite takes a guest as well as a machine
(NIBBLET_TEST_GUEST), so it covers Linux and Windows on Hyper-V, Windows
on KVM, and Linux and macOS on Apple Virtualization. Windows and macOS
images are installed through the API first, as Settings would, and the
browser test runs only where the guest has one. A VM's deletion is
checked on the host too.

Windows and macOS guests install the runner from GitHub at the release's
tag, which can't test unreleased code, so the server can name another
ref (NIBBLET_GUEST_REF), resolved once by the host for every hypervisor;
the suite gives it the commit under test, once GitHub has it.

Hosted runners have neither Hyper-V nor Apple Virtualization, so CI runs
Windows on KVM, in a workflow of its own given a WINDOWS_ISO_URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' first CI install ran three hours and timed out with nothing to
say why, since the only sign of progress was the VM switching itself off.
Setup and the bootstrap now say each step on COM1, and its failures; KVM
records COM1 while it installs, shows the last step as the image's
progress, takes a screenshot each minute beside the images, and calls a
power-off done only once the bootstrap said it was. CI keeps the screen
and logs when it fails, and stops waiting after 90 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows VM's overlay was made 32 GB, Linux's size, on an image
installed across 64 GB, so Windows booted onto a disk that ended in the
middle of C:. Its overlay is now its image's size.

QEMU's own output went nowhere once it daemonised, so a VM that stopped
by itself said nothing. It now runs detached with its output in its
folder, and a start that fails at once says what QEMU said. Windows
writes nothing to the console port, so a quiet console is instead what
QEMU said and a screenshot, which a VM whose agent never answers quotes.
The integration test stops waiting as soon as its nibblet is in error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add-Type compiled the two helpers inside PowerShell, with no output and
no time limit, and once sat there for good. They're now built by .NET
Framework's csc in a process of its own, each said on COM1, given ten
minutes, and failing with what the compiler said. Every failure the
bootstrap says starts the same, so KVM stops waiting at once and says
why, rather than after three hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Starting QEMU detached returned as soon as its control socket existed,
before its devices did, so the bridge could open a VM's agent port
early and the guest's agent then said hello twice: the second, in the
middle of a restore, cut it off. It now waits for QEMU to say it's
running over QMP, as daemonising did, and QMP's asking is a module of
its own, tested against a pretend QEMU.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A runner that stopped was described by the last three lines it said,
which for Node's own crash are its version and the entry's lines, never
the error, as Windows' first runs showed. The error line is now kept
when it's in the last forty.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows sat on "Building its serial helper" for most of an hour, past
the compiler's ten-minute limit, so the bootstrap itself had stopped,
or the guest had. It now says how long the compiler has run every half
minute, so the two look different, and KVM gives up when it's heard
nothing new for fifteen minutes, with a screenshot, rather than hours.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pnpm hard-links packages from its store, which for the bootstrap is
SYSTEM's own, and a hard link has its file's one set of permissions, so
the runner, as the agent's user, couldn't read them. They're copied
instead, taking C:\Nibblets' permissions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' install sat on "Is this the right country or region?": the
first-run screens skip region and keyboard only when oobeSystem says
them too, not just windowsPE. Meanwhile the bootstrap ran beneath them,
shut down mid-setup, and came back to do it again, which was every
"stuck" install so far. The answers are now in both, and the bootstrap
waits, saying so, until Windows calls its setup complete, failing with
where it's stuck after half an hour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows answered through its harness, then its snapshot ran past the
test's five minutes and held everything after it. The server now logs a
VM's home archive or restore as it goes, every half a minute, so slow
looks different from stuck; the tests print the server's log whenever
one fails, timeouts too, and give calls as long as the tests themselves
allow rather than fetch's five minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eaks

Windows' boot loader asks for a key for five seconds once the firmware
reaches it, and the install pressed one only for its first twenty, so a
slow host could miss it and leave the VM at an empty disk, silent until
the hours ran out. It now presses for two minutes, fails with a
screenshot if setup has said nothing in an hour, and CI keeps the
screen and logs of a cancelled run as well as a failed one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' home archive moved nothing for sixteen minutes, holding the
nibblet until the next test stopped its VM, and said nothing about why.
A VM's home archive or restore that moves nothing for three minutes is
now killed and fails, and a job that fails however it fails (stalled,
too slow, its machine gone) says the end of what it printed, which on
Windows names each file tar reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows lost touch with its runner forty seconds into a turn. Its COM
relay left the driver's 4 KB buffer, which drops what doesn't fit while
the agent's busy, and ended at any read error, which .NET raises for a
line error until it's cleared, taking every connection over the port
with it. It now has a megabyte each way, and says a read error and
carries on, ending only if they don't stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' sub-agent started and never called its model, and nothing could
say why: a sub-agent's output is only in its host's VM, and logs()
looked for a VM of its own. It now reads the sub-agent's run on its
host, and the server logs a VM sub-agent's last lines when it stops. The
test stops a sub-agent that hasn't answered, so its last words are in
the log it prints, with the child's status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pressing Enter for two minutes caught the DVD's boot prompt, then setup's
own progress page, where Enter is Cancel: Windows sat on "Are you sure
you want to quit?" at 24%. The install now looks at the screen every two
seconds and presses only while it's dark (the prompt, then the logo while
setup loads), stopping for good once setup's screen is up, so neither
Cancel nor a restart's prompt ever gets one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows' sub-agent died making a folder in its own temp folder, which
its ACL gave to "nibblet": the computer's name as well as the agent's
user, and Windows takes the name as the computer's. Folders are now
granted to the user's SID, looked up once, and new images' computer is
nibblet-vm so nothing else trips on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IAmJSD and others added 30 commits October 19, 2026 03:04
Making a nibblet on a Windows machine takes an optional product key.
It's kept with the nibblet and only ever sent back masked to its last
five, as Windows shows one. Whenever its VM starts, the server has the
guest install and activate it, unless it's activated with that key
already (an updated image is a fresh Windows), and says how it went in
the thread. The key reaches the guest on a job's stdin, and Windows'
licensing service in a SYSTEM process's environment: never a command
line, nor anywhere the runner can read it.

The create dialog shows the field for Windows machines, checking its
format as it's typed, and the mock now offers one. The Windows
integration run gives it Microsoft's published KMS client key, which
can't activate in CI, and checks it's tried and reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub's windows-2025 runners do run Hyper-V VMs, nested, and WSL 2, so
the Machines workflow now runs the integration suite there for both
guests: a NAT switch with DHCP for the VMs, Docker in WSL for the
stack's Linux containers and the Linux guest's disk, and the bridge on
Windows.

Two things that only worked on a Windows desktop with Docker Desktop
now work without: Hyper-V goes out through a named switch where there's
no Default Switch (Windows Server), set with --hyperv-switch, and a
VHDX Linux image is copied out of its tools container rather than
written to a mount, which Docker in WSL can't see. The test stack can
use Docker in WSL, translating the folders it mounts and reaching the
bridge on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The product key went to the guest as Readable.from([key]), whose chunk
is the string itself; the guest link framed it as no bytes, which threw
outside any handler and took the server down at a Windows nibblet's
first start. Keys go as bytes, the link sends a string chunk's bytes,
and a chunk it can't send fails its stream, not the server.

Hyper-V's capacity still asked for the switch it was given nowhere, so
it looked for one called nothing. And the Actions job built its runner
image with Windows' Docker CLI, which has no buildx, so the legacy
builder failed on COPY --chmod while the step passed: it now installs
buildx, and its steps stop at a failed command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linux on Hyper-V lost touch with its runner a minute into its first
turn, though its reply came a moment later: its agent's port is a COM
port, which stty left at the 9600 baud a serial port starts at, and
Hyper-V paces it to that. A COM port is now set to 115200, as Windows'
relay already was, with no modem control to take for a hang-up; a
virtio console (Apple's) has no speed and is left as it was.

The PowerShell line that stops a step at a failed command had gone
into the Windows-on-KVM job's bash build step too, which it ended at
once; it's in the Hyper-V job's alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It said it was stalled and carried on logging nothing moving for as long
as the VM took to come up, if ever: now it gives up once, and fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hyper-V's COM port lost what the host sent a Linux guest, so its runner
lost touch a minute into a turn, and a snapshot job never arrived. A
Linux guest on Hyper-V now listens on a Hyper-V socket instead (vsock,
through hv_sock), relayed by a small Python script that takes only the
host's connections, so nothing in the VM can pose as it; to the agent
it's a serial line, one stream whoever's connected. The bridge connects
through a helper it builds with Windows' own csc, which says once it's
connected, and registers the agent's service with Hyper-V where it can.
Windows guests keep their COM port for now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows guests on Hyper-V now listen on the agent's Hyper-V socket,
through nibblet-hvsock.exe, which the bootstrap builds beside the other
helpers: bound to the parent partition, so only the host gets in, and
turning away a connection from the VM itself. The agent picks it when
Windows says it's a Hyper-V virtual machine, and COM2 otherwise (KVM).
Every Hyper-V guest is then reached the same way, so its COM2 pipe goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows guest's image now makes the agent's user and signs it in at
boot, with nothing in the way (no first sign-in screens, lock screen or
sleep), and gets OpenComputerUse. runas --session starts a program in
that session, on the desktop a person sees, with the user's own
environment, so the guard starts the browser's OpenComputerUse there,
and the entry starts the desktop's, which the runner joins as a
sub-agent would. The browser is Edge, with its profile on the home disk
by policy, and the browser runs as the agent's user, as asked; its
OpenComputerUse is given none of the nibblet's own settings or secrets,
since that user can read its environment.

The AI's instructions now name its machine (not always a container),
and its guest's browser and how its desktop's apps are found. The app
launcher asks for a Wayland display on Linux alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its entry, guard and runner write to its machine, which nothing outside
it could read, so Windows' desktop not starting looked the same as any
other failure. nibblets.logs gives the end of it (a sub-agent's run, in
its host's), and the integration tests print it when one fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The entry started its desktop with `opencomputeruse`, which only the
guard's environment said to start through runas, and the guard's stdio
fallback left out runas's own arguments. What starts OpenComputerUse is
now passed to it, and goes before its arguments either way. Its Windows
build needs Visual C++'s runtime, which the image now installs.

Setup on KVM gets the install's whole time while its own screen is up:
on a slow runner it took over an hour to copy Windows. And the bridge
says when a guest's port drops, to find what cuts off a restore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s helpers in CI

Windows' firewall rules don't see loopback, which is how the tailnet
reaches a guest in Tailscale's userspace networking, so nibblet-fence
uses the Filtering Platform: a program may reach only the ports given.
A job builds every guest helper as the bootstrap does, and tries the
fence on real connections.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tailscaled runs as SYSTEM in userspace networking, on a pipe only an
administrator could have made, and its CLI as a hidden user of its own,
whose `up --unattended` makes it tailscaled's owner: tailscaled then
turns away the agent's user. nibblet-fence lets tailscaled reach only
the shared ports over loopback. The bootstrap unpacks Tailscale's own
signed installer, and runas finds each user's password in a file of its
own, for Tailscale's user as for the agent's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Windows guest on Hyper-V drops its agent's port now and then, and
nothing showed why: its agent's log went nowhere, and Hyper-V keeps no
console. The agent now writes its log to COM1 too, with what its relays
say (the hvsock relay saying when the host comes and goes), and the
Hyper-V plugin records COM1's pipe to console.log, as KVM does.

A KVM install said it was done, then started over: the install now
logs every line the bootstrap says rather than the last at each look,
and the bootstrap says which boot it started on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	apps/server/src/conductor.ts
…hat reboots

A Hyper-V guest's relay served one connection at a time. When the host
gave one up, the guest could go on waiting on it, half open, while each
new one waited unaccepted until Hyper-V reset it: about 30 seconds of a
dead port at a time, which cut off a restore. Both relays now take the
host's newest connection and drop the last, as the bridge does the
server's.

On KVM, Windows has twice started again just after saying its image was
done, losing what it last wrote. The bootstrap now flushes the registry
and C: before it says so, and the install turns the VM off itself if it
hasn't stopped a few minutes later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Connecting to a Hyper-V guest can take half a minute, and the relay it
was for may be closed by then, when the VM's been off a while and a new
one's been made for its next boot. The late connection was still
attached, and to a guest that takes the host's newest connection, it
replaced the live one: an automatic snapshot was cut off that way. Such
a connection is now dropped.

Linux's relay also lost its reader to a missing `global`, so nothing
from the host reached its agent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… they start

The bridge took any word "connected" on the helper's stderr for a
connection, and Windows' message for one that timed out says "the
connected party did not properly respond". Every failed attempt while a
guest booted looked like a connection that closed at once. It's now a
line of its own, and when the helper ends it says why, which the bridge
says with the port closing.

An archive or restore was metered from before its VM booted for it, so
a slow boot counted as a stall, and a meter given up on after its
pipeline was done threw with no one listening and took the server down.
They're metered from when the guest answers, and the meter's error is
handled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tunnel.ts takes isNotFound from the Docker plugin, where it lives here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new link said hello as it opened, and the relay said another as the
link attached; the guest answered both. The link opened on the first
answer and started its run, and the second, coming after, read as the
guest's port starting afresh, so the run was cut off. A restore whose
archive was small enough to start at once lost that race. The relay
now says hello only when the guest's port comes back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A job on a running VM (an automatic snapshot, say) went straight over
its link, so a stop that came meanwhile shut the VM down under it: the
archive was cut off, and the nibblet was left half stopped. A stop now
waits for the job, as it does for one on a VM booted for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A link and the bridge can both ask a guest to say hello as its port
comes up, and each answer after the first read as the port starting
afresh, cutting off the run the link had begun: a restore, most often.
Removing one asker didn't close the race. Now a link says hello with an
id of its own, which the bridge repeats when the port comes back; the
guest lets go of a link's channels only for a link it hasn't heard from,
and says an epoch that changes whenever it lets go of any, with an id
of its own so a restarted agent's never matches. A link counts its
channels lost only when the epoch changes, and doesn't hear another
link's answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Windows, OpenComputerUse starts through runas into the agent's
signed-in session, and runas waits for that session, which on a first
boot (its profile being made) takes minutes. It had 15 seconds to come
up, so on a slow host it was given up on before it could: the guard's
browser and the desktop alike. Through a wrapper it now has four.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Hyper-V restore is still cut off now and then with no sign why: the
link now says when it loses open channels (the guest saying hello again,
with which epochs, or the link closing), and the guest says when a new
link's hello drops the last one's.

On a slow runner, Windows' own first-run steps on KVM, which say
nothing, took over the 15 minutes an install could be quiet: now 30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and stream disks out of Docker's VM

A link opened on any hello, and a guest says one unasked as its agent
starts: the link took that, started its run, then heard its own answer
at a later epoch and called the run cut off. A link now opens only on
an answer to itself (or any, from a guest too old to say).

A VM's stop now waits for a job on it, and meanwhile its runner, still
up, reconnected and was taken back, so the stopped nibblet read as
idle. The conductor turns a stopped nibblet's runner away, and its
messages are heard only once its link is taken.

Where Docker's in a VM (on a Mac), making a Linux image wrote 32 GB of
file system through a mount of the host's folder, which took over half
an hour. The disk's now made inside the container and streamed out as a
sparse tar.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Its Windows build still wants Visual C++'s runtime, which the
bootstrap goes on installing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sn't in CI

A nibblet can be created with its resources, and the integration test
takes them from NIBBLET_TEST_CPUS and NIBBLET_TEST_MEMORY_MB, for a small
host.

Linux on Apple Virtualization was tried on macos-15-intel, the one
hosted runner with a hypervisor: its runner was lost each time, within
minutes of a nibblet's VM starting beside Docker's (Colima's), with
memory, disk and load to spare, so it runs one VM at a time. Depot's
Macs have no hypervisor. It runs on a Mac of one's own, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant