From aa93bae5d345dc6d1823c993b7ed6f77a2fea6a7 Mon Sep 17 00:00:00 2001 From: Indra Gunawan <1284811+coderbuzz@users.noreply.github.com> Date: Thu, 15 Oct 2026 17:27:33 +0700 Subject: [PATCH] docs: worker firewall applied on prod-heavy-1, SOP updated with the bridge rules Co-Authored-By: Claude Fable 5.1 --- audits/DOCKIFY-AUDIT-FABLE-5.1.md | 2 +- docs/vm-setup-sop.md | 32 +++++++++++++++++++++++++------ 2 files changed, 27 insertions(+), 7 deletions(-) diff --git a/audits/DOCKIFY-AUDIT-FABLE-5.1.md b/audits/DOCKIFY-AUDIT-FABLE-5.1.md index 8311ed5..461c54b 100644 --- a/audits/DOCKIFY-AUDIT-FABLE-5.1.md +++ b/audits/DOCKIFY-AUDIT-FABLE-5.1.md @@ -696,7 +696,7 @@ Code by Claude Opus 5.5 sessions, review and merge by Fable 5.1. Final `main`: ### Release 0.7.0 — done 2026-10-15 16:02 WIB - Tag `v0.7.0` on `b4d17dd`; CI published `dockify-linux-amd64` + `.sha256`. -- Controller: service stopped, DB snapshot taken (205 MB, `user_version` 0, integrity ok), `update.sh` verified the checksum and installed the binary; migration 0 → 4 took 21 s (32 duplicate route rows removed, 0 FK violations); health 200, UI through Caddy 200, 26 apps / 24 running, both workers' host keys pinned and their SHA256 fingerprints match the ones observed independently from the operator's machine. `DOCKIFY_TRUST_PROXY=true` added to `/opt/dockify/.env`. `DOCKIFY_HOST=127.0.0.1` was applied at 16:27 WIB after confirming that every real request and every GitHub webhook arrives through `dockify.amg.id` and that the only direct hits on `51.79.130.131:8080` were internet scanners (651 in 24 h, no login attempts; the VM has no firewall at all): port 8080 now refuses external connections, the domain and the webhook route still answer. A host firewall was then added (16:40 WIB): `ufw` with default deny incoming / allow outgoing and allow 22/tcp, 80/tcp, 443/tcp, 443/udp (Caddy HTTP/3), enabled at boot; verified with a fresh SSH login, the domain (200), and ports 8080/2019 blocked from outside. `install.sh` should default `DOCKIFY_HOST` to loopback and offer this firewall in the next release. +- Controller: service stopped, DB snapshot taken (205 MB, `user_version` 0, integrity ok), `update.sh` verified the checksum and installed the binary; migration 0 → 4 took 21 s (32 duplicate route rows removed, 0 FK violations); health 200, UI through Caddy 200, 26 apps / 24 running, both workers' host keys pinned and their SHA256 fingerprints match the ones observed independently from the operator's machine. `DOCKIFY_TRUST_PROXY=true` added to `/opt/dockify/.env`. `DOCKIFY_HOST=127.0.0.1` was applied at 16:27 WIB after confirming that every real request and every GitHub webhook arrives through `dockify.amg.id` and that the only direct hits on `51.79.130.131:8080` were internet scanners (651 in 24 h, no login attempts; the VM has no firewall at all): port 8080 now refuses external connections, the domain and the webhook route still answer. A host firewall was then added (16:40 WIB): `ufw` with default deny incoming / allow outgoing and allow 22/tcp, 80/tcp, 443/tcp, 443/udp (Caddy HTTP/3), enabled at boot; verified with a fresh SSH login, the domain (200), and ports 8080/2019 blocked from outside. `install.sh` should default `DOCKIFY_HOST` to loopback and offer this firewall in the next release. Workers (17:20 WIB): `prod-heavy-1` got the same `ufw` policy plus allow-in rules for `br-` and `docker0` (hairpin traffic), verified with a fresh SSH login, zero blocked packets from the Docker bridges, container-to-sibling-domain requests still answered, and all apps reachable; MongoDB 27017 (Docker-published, used by an external client at 103.11.186.107) is unaffected by ufw. `staging` keeps Photon OS's own iptables policy (INPUT DROP, 22 only), which already matched. - Verification redeploy of `kv-dev.amg.id` (staging): success in 3 s, container env byte-identical to before (8 vars), `.env` now 0600 and quoted, Caddy routes persisted on disk (7/7), HTTPS unchanged. - Rollback (one command, as root on the controller): `sudo /opt/dockify/rollback/rollback.sh` restores the 0.6.22 binary, the pre-update DB snapshot and the original `.env`, and keeps the 0.7.0 database aside. Delete `/opt/dockify/rollback` (≈220 MB) once 0.7.0 has run for a few days. - Still open after the release: delete the dead server `production-1` from the UI (now possible); optionally remove `~/.ssh/dockify*` on both workers; B14/B11 and the Settings-page self-update (`systemd-run`) remain unexercised on the real hosts. diff --git a/docs/vm-setup-sop.md b/docs/vm-setup-sop.md index fb5f975..75b6b5f 100644 --- a/docs/vm-setup-sop.md +++ b/docs/vm-setup-sop.md @@ -260,13 +260,33 @@ Log out and in again so the `docker` group applies. ls -la /opt/dockify/caddy/config.json # persisted config ``` -### 2.5 Host firewall on the worker — [recommended], not yet applied on production workers +### 2.5 Host firewall on the worker — applied on `prod-heavy-1` 2026-10-15; `staging` (Photon OS) uses the distribution's iptables policy instead -Same procedure and watchdog as §1.4: `ufw default deny incoming`, allow `22/tcp`, -`80/tcp`, `443/tcp`, `443/udp`. Two Docker caveats: Docker publishes ports with its -own iptables rules that bypass ufw, so anything a compose file publishes with -`ports:` is reachable regardless; and the Caddy admin port is already bound to -loopback. Prefer `expose` (simple mode does this) over `ports` for apps. +Same procedure and watchdog as §1.4, plus two rules that keep container-to-host +traffic working (an app calling a sibling app through its public domain reaches +the host's own IP through `docker-proxy`, which arrives on the bridge interface): + +```bash +BR=br-$(docker network inspect dockify -f '{{.Id}}' | cut -c1-12) +sudo ufw default deny incoming && sudo ufw default allow outgoing +sudo ufw allow 22/tcp && sudo ufw allow 80/tcp && sudo ufw allow 443/tcp && sudo ufw allow 443/udp +sudo ufw allow in on "$BR" comment 'dockify bridge hairpin' +sudo ufw allow in on docker0 comment 'docker bridge' +``` + +Before enabling, scan what is actually open (`ss -lntup`, `docker ps --format +'{{.Names}} {{.Ports}}'`) and keep any host-level port other services rely on. +Docker caveats: ports a compose file publishes with `ports:` are reachable through +Docker's own iptables chains regardless of ufw (on `prod-heavy-1`, MongoDB 27017 +stays public this way; restrict it with a `DOCKER-USER` rule to known client IPs, +not with ufw); the Caddy admin port is bound to loopback. Prefer `expose` (simple +mode does this) over `ports` for apps. After enabling, check +`journalctl -k | grep 'UFW BLOCK'`: there must be no `SRC=172.` (bridge) entries. +The dockify SSH user needs passwordless sudo for this (`/etc/sudoers.d/dockify`, +mode 440, validated with `visudo -c`). On Photon OS (`staging`) `ufw` is not +available; its default `iptables.service` policy (INPUT DROP, 22 only, hairpin +rule for the dockify bridge, persisted in `/etc/systemd/scripts/ip4save`) is +equivalent and must be kept. ### 2.6 Conventions enforced per app (by Dockify, nothing to configure)