Skip to content

feat: tools workloads, maintenance and monitoring - #10

Merged
Chalvin96 merged 14 commits into
masterfrom
feat/tools-vm
Oct 25, 2026
Merged

Chalvin96 merged 14 commits into
masterfrom
feat/tools-vm

Conversation

@Chalvin96

@Chalvin96 Chalvin96 commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Changes

  • Add Penpot and private-repository CI runners with separate rootless Docker accounts, migration recovery and optional VM provisioning.
  • Add scoped CI cleanup, Penpot health checks and Proxmox resource/storage monitoring.
  • Provision Grafana alerts, concise Discord notifications and Mailrise notification roles.
  • Keep Caddy administration on loopback while exposing a dedicated metrics endpoint.
  • Whitelist local-network access for applications and separate GlitchTip from observability services.
  • Restrict credential-file permissions, suppress secret template output, upgrade Grafana and reject plaintext private configuration in CI.
  • Add scoped deployment tooling and recreate Prometheus when replacing its configuration.
  • Share Docker installation and Grafana provisioning, keep one CI cleanup scheduler, and remove unused legacy assets.

Validation

Ansible lint, formatting and secret scans pass. Local regression checks pass for
backup failure handling, migration recovery, CI cleanup, health collection,
notification templates, secret protection, shared monitoring rendering and native application-firewall checks. Real configuration remains encrypted;
examples and documentation use placeholders. No infrastructure migration is required
to review this PR.

…ntry point

Tools VM (Proxmox, SSD boot + HDD data at /srv):
- docker: Docker CE, log rotation, weekly prune timer, node-exporter
- penpot: Penpot stack on /srv with nightly pg_dump + assets backup
- github-runner: per-repository self-hosted Actions runners (private repos
  only), registered via a fine-grained PAT from the vault

Edge node:
- notify: Mailrise SMTP -> Apprise -> Discord gateway in a pinned venv,
  hardened systemd unit, LAN-only nftables table loaded without touching
  the global ruleset
- caddy: render `caddy_sites` into the validated Caddyfile (GlitchTip at
  sentry.<domain>)

Operations:
- run.sh: uv-pinned Ansible, collections install, vault password from
  .vault_password or a single RAM-only prompt; edit-config/edit-vault/
  bootstrap/lint subcommands
- Commit the inventory (addresses come from config) and a vault-encrypted
  config.yml; config.yml.example documents every variable, nodes first
- bootstrap: authorize `deploy_authorized_keys` per control machine
- site.yml: per-role tags for scoped runs (e.g. --tags notify)
…host

Review fixes:
- penpot: never print the .env secrets (no_log + diff: false)
- docs: bootstrap with `-e ansible_user=<user>`; the inventory's
  ansible_user=deploy overrides `-u`
- check mode on a fresh host: skip enable/start steps and handlers whose
  unit or file is only created in a real run; read dpkg arch in check mode
- notify: include the mailrise table in /etc/nftables.conf so a global
  reload by another role re-creates it (no reload triggered here)
- github-runner: pass the registration token via ACTIONS_RUNNER_INPUT_TOKEN
  instead of argv; tolerate an empty repo list
- site.yml: force_handlers on every play
- penpot backup: OnFailure alert to Mailrise; remove partial files on error
- run.sh: trap covers the plaintext config as well as the password file,
  refuse a .vault_password that is not 0600, lint uses the CI pins,
  clearer names, and a leading flag (./run.sh --check) means all hosts
- README: state that runner jobs can read Penpot secrets and data

Found by running the notify role against a disposable systemd Debian 12
container (check mode fresh, real run, idempotent re-run, check mode
provisioned):
- pip module needs python3-packaging on the target
- DynamicUser= collided with the static mailrise group (217/USER); use a
  static system user and verify the SMTP port after start
- Mailrise appends @mailrise.xyz to bare keys; render full
  <channel>@<notify_mail_domain> addresses
@Chalvin96 Chalvin96 changed the title feat: tools VM roles, Mailrise notifications, clone-and-run feat: tools workloads, maintenance and monitoring Oct 25, 2026
@Chalvin96
Chalvin96 merged commit 5beb0a6 into master Oct 25, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant