Skip to content

Add Proxmox VE LXC runtime backend - #34

Open
philip-ulrich wants to merge 10 commits into
calagopus:mainfrom
philip-ulrich:codex/pve-lxc-runtime
Open

philip-ulrich wants to merge 10 commits into
calagopus:mainfrom
philip-ulrich:codex/pve-lxc-runtime

Conversation

@philip-ulrich

@philip-ulrich philip-ulrich commented Sep 29, 2026 •

Copy link
Copy Markdown

Adds a native Proxmox VE 9.2 LXC runtime so Wings can create, adopt, run, stop, replace, and destroy game-server containers without Docker. Runtime selection remains behind ServerExecutor: auto prefers reachable Docker, otherwise chooses local PVE, and persists that decision so installing Docker later does not move existing workloads.

The PVE path maps panel CPU, memory, storage, allocations, mounts, devices, console, installation helpers, networking, firewall policy, backups, and cgroup telemetry onto native LXC primitives. It uses node-scoped ownership tags, retries advisory VMID allocation collisions, refreshes mutable OCI tags by resolved digest, bounds every local command and task/config-helper wait, waits for stop completion before destroy, scopes wildcard ports by address family, and skips transiently disappearing containers during used-port discovery.

OCI template downloads are serialized without blocking unrelated container provisioning. Installer status files are owner-writable rather than world-writable, helper log readers recover from truncation, and static networks no longer receive the host-managed DHCP marker.

Firewall policy remains panel-authoritative. When the datacenter firewall is enabled, Wings writes only its tagged VM rules and source-file IP sets, preserves administrator rules, uses the panel's default-allow semantics, and emits a protocol/port-free terminal drop for “deny everything else.” auto clears stale Wings guest rules before falling back to host nftables or iptables. Firewall reload and reconciliation now hold one operation lock, including cleanup of the remembered VMID.

Native Tundra integration uses pid:<init-pid> references. Wings requires the connected daemon to advertise process_container_refs and fails closed when the capability is absent. The runtime-neutral resolver now lives as source on the codex/process-container-refs companion branch instead of an out-of-tree patch in this repository.

Operator documentation is integrated into the existing Wings installation, configuration, and private-network sections on the codex/pve-lxc-runtime-docs website branch. Known initial limitations are documented there: PVE does not currently provide Docker-equivalent exit/OOM metadata; panel entrypoint overrides, CPU pinning, unlimited memory/swap, digest-form image references, and private-registry credentials are rejected explicitly.

Validation:

  • rebased onto current main; review follow-ups are preserved as separate commits
  • cargo fmt --all -- --check
  • production cargo clippy -p wings-rs --bin wings-rs -- -D warnings
  • 774 tests passed with RUSTFLAGS="-D warnings"; 11 Docker-socket integration tests ignored
  • companion Tundra branch passes formatting, Clippy, and all workspace tests (300 passed; 4 root/Docker-only tests ignored)
  • release build from 9c9afe6 installed on a PVE 9.2.2 test node running Wings 1.2.4 (sha256:4ebc68a47fad1caf3a66bef41926ecf729c633ae5731c83ab196d32ab4782c16)
  • existing Minecraft LXC remained running across daemon restarts
  • live disposable smoke test passed OCI create/start/stop/hard-stop, runtime config, DHCP, idmap, procfs, replacement, persistence, destroy, and cleanup contracts
  • Tundra advertised process_container_refs and adopted the process-backed server reference
  • PVE guest policy contained TCP/UDP 25565 accepts followed by an unrestricted drop
  • public TCP/25565 remained reachable through the edge tunnel

cargo clippy --all-targets -- -D warnings is not currently a repository gate because pre-existing test modules contain hundreds of denied unwrap/indexing/assertion lints; the production binary target is clean.

@0x7d8 0x7d8 linked an issue Sep 29, 2026 that may be closed by this pull request
@philip-ulrich

philip-ulrich commented Sep 29, 2026 •

Copy link
Copy Markdown
Author

Still testing, fixing, and adjusting things, but this should be most of the way there. I’m not comfortable taking it out of draft yet, but you’re welcome to start reviewing and providing feedback. I’ll stop amending existing commits so it’s easier to see what changed since your last review.

A few notes beyond the PR description:

  • I added a general pull request test workflow. It is not specific to the LXC backend and could be split into a separate PR if preferred.
  • The current CI validates formatting, the Tundra patch, and the Rust test suite, including 60 new PVE specific tests. Live PVE testing is still manual because GitHub-hosted runners do not provide a Proxmox environment.
  • Live testing so far has been on one PVE 9.2.2 node with one persistent Minecraft server plus disposable lifecycle tests. > PVE 9.1 is required because that's when docker images were supported. Multi-node operation, larger-scale concurrency, GPU/device workloads, and more game types still need broader testing.
  • The implementation currently targets the local PVE node and unprivileged LXCs. It is not attempting migration or remote-node orchestration yet.
  • The PVE implementation files are still large. I expect some separation into smaller runtime, networking, installation, and API modules before this is ready to merge.

One other thing worth noting, I intentionally left everything "deployment method agnostic" so that we don't have to make any panel changes. There are things that could be cleaned up a little bit if we added some panel controls for pve vs docker. But it was written initially for me so that I could get it working without having to maintain two forks if you didn't want it upstream. 😄

@philip-ulrich
philip-ulrich marked this pull request as ready for review September 30, 2026 00:31
@philip-ulrich

Copy link
Copy Markdown
Author

Should be good now. Might have follow up PR(s) in a week or so. We'll be moving 50+ servers into the panel with more to come after. We should quickly find issues if they exist.

Wings now allocates Proxmox container IDs at or above
runtime.pve_lxc.min_vmid (default 200) so auto-created guests stay
clear of manually created infrastructure containers below the floor.
When /cluster/nextid points below the floor, the allocator scans the
cluster inventory across all guest types for the first free ID.
@0x7d8 0x7d8 self-assigned this Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PVE LXC Support

2 participants