This file provides guidance to AI coding assistants (Claude Code, Copilot, etc.) when working with code in this repository.
Verify load-bearing factual claims — code, tooling, third-party behaviour — against the code, the repo docs, or a web search before stating them. Flag anything unverified, or a deduction.
Badge in chat output only (never in commit messages, code comments, or committed files), inline
right after the claim, with a path:line cite for code/doc:
✅ code/✅ docpath:line— verified in the code / repo docs✅ webURL — verified online (bare URL, outside the code span; pin to a commit SHA, not a branch)💭 deduction— inferred from verified facts⚠️ unverified— unverifiable, or from training data
Skip badges on restatements, tool output, and descriptions of your own next step.
virtkit — a rootless microVM toolkit shipped as static-musl binaries (vk + the
embedded vk-agent, plus the optional vk-registry central server, the
vk-runnerctl runner throttle, the experimental vk-hub fleet hub and local web UI, and
the experimental vk-gitlab runner that has the hub run GitLab jobs).
It boots OCI/Docker images as fast microVMs on its embedded
libkrun VMM, gives them a shared LAN with egress
over ordinary host sockets (no tap, no bridge, no CAP_NET_ADMIN, no root), and drives
commands into them over vsock.
The same codebase powers local compose-service VMs and a GitLab custom executor. See
README.md for the full feature tour.
A Cargo workspace (Cargo.toml, edition 2024) with fifteen crates:
vk-core/— the shared host↔guest library: the wire protocol (messages,framing,addr,net,status,fleetctl), the formats both sides speak (atop,oomkills), plus the runtime helpers both sides build on (exec,forward,pty,dockerignore). Deliberately free of guest-only concerns and of russh, so the host links none of that.vk-driver/— the host driver (depends onvk-core): image building/conversion (OCI → ext4/initramfs), the compose service runner + control plane, the GitLab executor, the userspace L2 network switch (ARP/DHCP/DNS + transparent TCP/UDP egress viaipstack), and the libkrun VMM (vmm/libkrun_sys; the pinned guest kernel and vk-agent are embedded sovkruns self-contained).vk-agent/— the guest PID 1 / agent (depends onvk-core): brings a systemd-less guest up (mounts, networking, hostname, virtio-fs, optional SSH) and serves an exec channel overvsockso the host can run commands inside the VM.vk-registry/— the content-addressed OCI store (lib) plus a standalone server (bin).vk-driverlinks the lib for its in-process build cache. Thevk-registrybinary is a central OCI-distribution daemon meant to be shared by every runner: it serves the store over HTTP(S), fronts upstream registries as a pull-through cache (caching only digest-addressed content), coordinates build-once via a lease/heartbeat lock API (/lock), and is a backend for thetaskbuild cache. See itsDESIGN.md.vk-selfupdate/—<tool> updatefor the binaries a user installs by hand: resolve a GitHub release, check the download against the digest published beside it, smoke-test it, and rename it over the running binary. Parameterized by tool so every one of them passes the same gates; the caller supplies only which asset it is and what version it was built as.vkand thevk-registrybinary are the callers.vk-fs/— filesystem objects created private and published whole: mode asked for at creation rather than left to the process-wide umask,renameinto place so a name never leads nowhere, a directory resolved once and worked through its descriptor, and a name acted on only where it cannot have become another user's. A leaf with no dependencies butlibcandanyhow, so every crate here can use it —vk-runnerctlincluded.vk-hub-proto/— whatvktells other programs on its host, such asvk-hub: the list of the host's VMsvk workloadsprints; and whatvk nodeandvk-hubexchange in fleet mode: enrollment, the session's messages, protocol version negotiation, and the payloads each side signs or verifies; and the hub's client API and the job spec a producer such asvk-gitlabsubmits through it; and whatvk-registryandvk-hubexchange when the hub manages a registry, andvk-gitlabwhen it enrolls as a relay. Types and pure functions only — no runtime, transport or crypto.vk-peer/— what a peer ofvk-hubneeds to join it and hold a session: its ed25519 identity, enrollment, TLS and the session's opening handshake, release keys, and the pieces of a self-update on trial (download under the peer's signature, publish by rename,fexecvewithalarm(2)armed).vk nodeuses it; each peer keeps its own messages and state.vk-oidc/— the OIDC relying partyvk-registryandvk-hubsign people in with: discovery checked against the configured issuer, the authorization URL withstateand PKCE, the code exchange and UserInfo, and the checks on an[oidc]table's issuer and client-secret file. Each caller keeps its own routes, cookies and sessions. Itsfake-idpfeature is an in-process provider for their tests.vk-hub/— the hub (bin, experimental).vk-hub serveis the fleet hub: it enrollsvk nodes with single-use tokens, pins each node's ed25519 key, holds their WebSocket sessions and keeps their inventory and heartbeats in a redb database, along with the desired state and commands — drain, quarantine, update, reset — it steers them with; it holds thevkreleases they update to, and rolls a release out a wave at a time.vk-hub servealso serves a web UI of the fleet whenui_addris set;vk-hub localserves the same UI for the VMs of the machine it runs on, read fromvk workloads --watch. Both sign people in with single-use links and keep an audit log. Built like thevk-registryserver (hyper, rustls on ring); operators reach it through a private unix socket and the web UI. Seedocs/fleet-design.mdanddocs/fleet-prototype.md.vk-gitlab/— the GitLab runner for a fleet (lib + bin, experimental): gitlab-runner's GitLab-facing side, reimplemented from gitlab-runner v19.5.0 (MIT, seeNOTICE). It asks GitLab for jobs only once the hub has reserved room, translates each intovk-hub-proto's job spec, submits it through the hub's client API, and copies the job's output and outcome back into GitLab's trace and state; a state file per runner lets a restarted daemon pick its jobs up again. Its tests run against an in-process fake GitLab and fake hub, with fixtures ported from gitlab-runner's own suite. Seedocs/gitlab-dispatch.md.vk-runnerctl/— the only component that runs as root, and deliberately the smallest: it sets gitlab-runner'sconcurrentfrom a number unprivilegedvkleaves in a file, clamped into a range only root can configure. It takes no arguments and no paths from its caller, so granting itNOPASSWDgrants nothing else; all the policy lives invk. Its config edit is also a library, for unprivilegedvkto apply to a runner config it owns.vk-tpm/— the TPM 2.0 engine in Rust behind libkrun's TPM CRB device (Windows guests'tpm=on): command processing, sessions and parameter encryption, hierarchies and dictionary-attack protection, PCRs, hash sequences, objects and keys (RSA, ECC P-256, primaries derived as libtpms derives them), contexts, persistent objects, NV indices, policies, attestation, duplication, EK provisioning, versioned state; crypto from RustCrypto crates only. Seedocs/tpm-design.md.vk-uefi-vars/— the UEFI variable service of Windows guests, on the host behind libkrun's uefi-vars device: the variables in an edk2 variable store image, GetVariable, GetNextVariableName, SetVariable and QueryVariableInfo as edk2's SMM variable driver answers them, Secure Boot's authenticated writes (PKCS#7, signature lists), private authenticated variables, variable policies and locks; crypto from RustCrypto crates only. Seedocs/uefi-vars-design.md.vk-testkit/— helpers the end-to-end tests (tests/*.sh) run on the host, so a test needs nothing of it but a shell andvk:git-repowrites a bare repository git's dumb HTTP transport can clone. Test-only, never shipped.
libkrun is vendored (its own cargo workspace, locally patched) under
third_party/libkrun — see its VENDOR.md for the patch list. Its devices crate depends on
vk-tpm and vk-uefi-vars by path, so its Cargo.lock locks their dependencies too: after
changing them, run cargo update -p vk-tpm (or -p vk-uefi-vars) in third_party/libkrun as
well. CI lints and tests that workspace separately (quality.yml's libkrun job); locally,
point ./dev.sh at it with --manifest-path third_party/libkrun/Cargo.toml, e.g.
./dev.sh test --manifest-path third_party/libkrun/Cargo.toml -p krun-devices --lib cmos.
The guest kernel is a vanilla Linux vmlinux built from a vendored config fragment
(kernel/); it is pinned and built separately from the binaries.
The release artifacts are built reproducibly inside a pinned devcontainer image
(.devcontainer/Dockerfile: the nixos/nix base by digest, the toolchain from
.devcontainer/nix/flake.nix pinned by flake.lock). Nix runs only inside that image at
build time — no Nix on the host. A musl cross gcc compiles the C our deps vendor (ring,
zstd, jemalloc) and links the static-musl binaries; no system C libraries are linked.
Release build scripts can use Docker or vk, so no local Rust setup is needed. The
fast edit loop below is deliberately vk-only and never invokes Docker.
./build-kernel.sh [--no-cache] # guest kernel vmlinux -> dist/ (vk or Docker; slow) — run first
./build-firmware.sh [--docker|--no-cache] # UEFI firmware CLOUDHV.fd -> dist/ (optional; embedded by build.sh)
./build.sh # static-musl binaries -> dist/ (vk or Docker)
./build.sh --fast # same, but the debug profile -> much faster iteration
./build.sh --only=vk-hub # compile just these binaries (comma list), not assembled
./build.sh --assemble # append dist/{vk-agent,vmlinux,CLOUDHV*.fd} to dist/vk-bare -> dist/vk + vk.sha256 + build-info.txt
./dev.sh check # type/borrow checking, whole workspace, every target
./dev.sh clippy # the lints CI gates on, same scope
./dev.sh fmt [--check] # rustfmt the workspace with the pinned toolchain
./dev.sh test # the whole test suite, doctests included
./dev.sh check -p vk-core # the same, narrowed to one affected crate
./dev.sh test -p vk-core --lib … # one module's unit tests (see below)
./dev.sh build -p vk-testkit # debug binary of a test helper (not `vk`: see build.sh)
./dev.sh shell # interactive shell in that same VM
./audit.sh [--deny warnings] # cargo-audit against the committed Cargo.lock
./sweep.sh [--time 15] # cargo-sweep stale target/ artifacts (default --installed)
./update.sh # bump the pinned Rust toolchain + re-lock the flake
./update-kernel.sh [--lts|--stable] # bump the pinned guest kernel (defaults to LTS)
./update-ci-tools.sh [--latest] # re-pin vk-hub's built-in CI tools (vk-hub/ci-tools/)When you need a runnable binary, use ./build.sh --fast (alias --debug). It builds the
unoptimized debug profile — a much faster compile that still produces the same static-musl
vk with the kernel + agent embedded. The output is not stripped and not reproducible, so it
is for iterating only, never a release artifact (it cannot combine with --bootstrap-check).
Use a plain ./build.sh for anything you ship or compare bytes against.
Every local (non-release) build also gets line-tables-only debuginfo from [profile.dev],
which keeps file:line in panics and backtraces while dropping the bulky per-variable
DWARF; dependencies get none at all. When you need a debugger, build with cargo's
--profile debugging (./dev.sh check|test accept it too) — build.sh offers only the
release and dev profiles.
build.sh compiles dist/vk-bare, then assemble.sh appends dist/vk-agent and
dist/vmlinux to produce dist/vk. When present, dist/CLOUDHV.fd and its
CLOUDHV_VARS*.fd templates join the payload, read at runtime by vk-driver/src/embed.rs. Both
./build.sh and ./build.sh --fast embed dist/vmlinux and fail when it is absent,
so ./build-kernel.sh comes first in a fresh checkout; --no-kernel builds a vk without
the embedded kernel instead (it then needs --kernel at runtime, and is not shippable).
Only the assembling needs the kernel: ./build.sh --only=vk,vk-agent compiles without it.
During iterative edits, do not run cargo build or release builds: ./dev.sh build is
for a non-shipped helper such as vk-testkit, and a runnable vk comes from
./build.sh --fast. Start with ./dev.sh check -p <affected-crate> for type and borrow
checking. Then run only the test target and module affected by the change, for example:
./dev.sh test -p vk-core --lib dockerignore::tests
./dev.sh test -p vk-core --test exec disconnect_kills_remote_process
./dev.sh test -p vk-driver --bin vk atop_view::tests./dev.sh clippy -p <crate> runs CI's Clippy gate for one crate. It defaults to
-- -D warnings, and explicit -- lint flags replace that default.
With no arguments, each mode covers the whole workspace. check and clippy add
--all-targets, so test code receives the same coverage as CI's Clippy run; fmt adds
--all; test keeps Cargo's defaults to include doctests. Run the broader commands
before committing. Keep the narrow forms for the edit loop itself — the wide ones take
minutes where a scoped run takes seconds.
dev.sh is Docker-free: on first use it boots the pinned build image as a shared
vk development VM; later invocations use vk exec, avoiding another image build and
boot. It reuses target/ — its RUSTFLAGS match build.sh's exactly, so it shares
dependency artifacts with ./build.sh --fast — and rejects optimized invocations. The
VM powers itself off after half an hour with no cargo command or open shell, so a
forgotten one stops holding memory; ./dev.sh stop ends it immediately. Starting a VM
also deletes the stopped VMs of this repository's checkouts that no longer exist. A
vk and a flock on PATH are required and there is deliberately no Docker fallback.
VK_DEV_CPUS and VK_DEV_MEM size the VM, VK_DEV_IDLE_SECS sets that idle window
(0 keeps the VM until it is stopped). Use ./build.sh --fast only when an executable
is actually needed for runtime testing. ./dev.sh shell opens an interactive shell in
that VM (same user, directory and cargo environment) and holds the VM for as long as
the shell runs, for the odd cargo or toolchain command the modes above refuse.
That VM boots with nested virtualization where the host allows it and the vk on
PATH can ask for it, so the shell can also run what it builds: ./dist/vk run …
works inside it, as does ./build.sh --fast --use-virtkit=./dist, their images and
boot scratch redirected under /var/tmp/vk on the guest's own disk rather than the
RAM-backed /tmp. To search the tree from inside the VM, reach for ugrep and bfs;
the image's grep and find are busybox applets.
The toolchain is pinned in rust-toolchain.toml (musl target, clippy + rustfmt). Run
cargo directly if you have it, or inside the devcontainer image to match CI exactly
(clippy compiles the workspace, so it needs build.sh's writable cargo home — see
.github/workflows/quality.yml). These are the CI-parity commands, run to verify a
change; the edit loop above is what to use while iterating.
cargo build --release --workspace
cargo test --workspace # tests, e.g. vk-core/tests/exec.rs
cargo fmt --all # prefer ./dev.sh fmt: pinned rustfmt
cargo clippy --workspace --all-targets --locked -- -D warnings- Rust: rustfmt + clippy, pinned via
rust-toolchain.toml(edition 2024). CI runscargo fmt --checkandcargo clippy ... -D warnings. - Shell: Bash,
set -euo pipefail. Scripts that also run inside the build image (e.g.audit.shunder CI) must stay POSIX-compatible — assume onlyshthere. - Dependency audit:
cargo-auditwith the RUSTSEC ignore list in.cargo/audit.toml(each entry documented with rationale + residual risk).
- GitHub Actions (
.github/workflows/):ci.yml(lint + test + audit + build on push/PR),release.yml(gate pushes torelease, includingtests/release-e2e.shagainst the builtvk, then fast-forwardmain, push thev*tag and publish the GitHub release; seedocs/releasing.md), with reusablequality.yml(lint + test + audit),build.yml(the kernel, the firmware and each binary in parallel jobs, then the job that assemblesvk) andrebuild.yml. A release is attested by rebuilding each binary from scratch in avkmicroVM and checking it matches the Docker build byte for byte (the per-binary form ofbuild.sh --bootstrap-check). - End-to-end suite (
tests/):VK=./dist/vk tests/release-e2e.shruns it against a local build — it needs KVM, e2fsprogs and network — and naming scripts narrows the run. Every othertests/*.shis picked up by its glob, so adding one gates the next release with no registration step — exceptwindows-*.sh(run withISO_DIR) andgitlab-e2e.sh, which boots a GitLab CE of its own to testvk-gitlab(run withE2E_GITLAB=1; seedocs/gitlab-e2e.md). Helpers live in subdirectories (tests/fleet/,tests/gitlab/), out of the glob's reach.
Reproducibility is load-bearing: the binaries are baked into microVM images. Keep
builds byte-deterministic (pinned toolchain/base image, SOURCE_DATE_EPOCH, path
remapping). Do not break the pinning when changing build inputs.
See .agents/commit-message-guidelines.md for
the format, scope list, body rules, and changelog rules. In short: one concern per
commit, independently buildable; single-line imperative summary (no trailing period)
with an optional scope: prefix (e.g. ci:, build-kernel.sh:); a wrapped body only
when the diff does not speak for itself, kept high-level. A user-visible change updates
CHANGELOG.md in the same commit, pitched even higher-level than the message.
Code review is expected on the production branch (main): one concern per commit, every
commit independently buildable, and every changed line auditable at a glance. Review
against the conventions in
.agents/coding-guidelines.md and the message rules in
.agents/commit-message-guidelines.md.
See .agents/coding-guidelines.md for general
conventions, formatting requirements, and per-language guidelines (Rust, Shell).