packaging(ns8): wire up the robot suite

.github/workflows/test-module.yml

@@ -0,0 +1,84 @@
+name: Test module (robot)
+
+# Robot Framework integration tests for the NS8 module. The suite in
+# packaging/ns8/tests installs the module on a real NS8 cluster, configures it,
+# checks the route, screenshots the UI and removes it again.
+#
+# WHY THIS DOES NOT USE THE STOCK NS8 WORKFLOW
+#
+# NethServer/ns8-github-actions provides test-module.yml@v1, the one-line
+# entry point every NS8 module uses. It cannot be used here. It derives the
+# image under test from the repository name:
+#
+#   name  = basename(repository), with any leading "ns8-" stripped
+#   owner = lowercased repo owner
+#   tag   = head branch, with "main" forced to "latest"
+#   image = ghcr.io/<owner>/<name>:<tag>
+#
+# For this repository that yields ghcr.io/calnode/calnode:latest, which is two
+# wrong answers at once:
+#   * the module image is ghcr.io/calnode/ns8-calnode, not ghcr.io/calnode/calnode
+#     (the latter is the Calnode application image, which has no NS8 module
+#     manifest, so add-module would fail);
+#   * this repo deliberately never publishes a :latest module tag. :latest stays
+#     pinned to releases, and main publishes :edge, exactly as docker-publish.yml
+#     does for the app image.
+#
+# So this workflow calls test-on-digitalocean-infra.yml@v1 directly, which is
+# the reusable workflow test-module.yml delegates to anyway, and supplies the
+# image URL itself via the `args` input. Everything else (DigitalOcean
+# provisioning, NS8 install, create-cluster, the test runner, artifact upload,
+# tmate debugging) is still theirs and unmodified.
+#
+# NOT RUNNING AUTOMATICALLY YET
+#
+# It needs a `do_token` repository secret (DigitalOcean API token) for the
+# cluster provisioning, and this repo does not have one. Declaring the workflow
+# with a workflow_run trigger now would just fail red on every publish, so it is
+# manual-only until that is sorted out.
+on:
+  workflow_dispatch:
+    inputs:
+      image:
+        description: 'Module image under test'
+        required: false
+        type: string
+        default: 'ghcr.io/calnode/ns8-calnode:edge'
+      run_ui_tests:
+        description: 'Run the [ui] tests (slower, needs the Playwright image)'
+        required: false
+        type: boolean
+        default: false
+      debug_shell:
+        description: 'Hand out a tmate debug shell on failure'
+        required: false
+        type: boolean
+        default: false
+# To run this on every successful publish instead of by hand, add:
+#   workflow_run:
+#     workflows: ['NS8 module publish']
+#     types: [completed]
+# ...after setting the do_token secret.
+
+jobs:
+  test:
+    name: Run tests
+    permissions:
+      contents: read
+      pull-requests: write
+      statuses: write
+    uses: NethServer/ns8-github-actions/.github/workflows/test-on-digitalocean-infra.yml@v1
+    with:
+      # packaging/ns8 holds both the module definition and tests/, so this is the
+      # module root for the purposes of the runner.
+      path: packaging/ns8
+      # `args` is the second positional argument to run-ns8-tests, i.e. the
+      # IMAGE_URL that the suite passes to add-module. See the header note.
+      args: ${{ inputs.image }}
+      # One leader node. The default is "dn1 rl1" (two droplets); the suite is
+      # single-node, so a second one only doubles the cost.
+      leader_nodes: rl1
+      run_ui_tests: ${{ inputs.run_ui_tests }}
+      debug_shell: ${{ inputs.debug_shell }}
+    secrets:
+      do_token: ${{ secrets.do_token }}

.gitignore

@@ -62,3 +62,6 @@ pnpm-debug.log*
 # Vitest browser-mode artifacts (failure screenshots / attachments)
 __screenshots__/
 .vitest-attachments/
+
+# NS8 Robot Framework test output (output.xml, log.html, report.html, screenshots)
+packaging/ns8/tests/outputs/

packaging/ns8/README.md

@@ -114,8 +114,11 @@ byte-identical across reconfigures, `remove-module --no-preserve` with
 automatic route cleanup, and a full cluster backup → destroy → restore cycle:
 after restoring, the app served 200, logged "keyvault: DEK unwrapped from
 keystore", the encryption-key fingerprint was unchanged, and an event type
-created before the backup was still there. Still untested live: the
-automated `tests/` robot suite, which needs a test cluster.
+created before the backup was still there.
+
+The `tests/` robot suite is wired up and dry-run clean (5/5, Robot 7.5), but it
+has still never been executed against a live cluster. Every live check above
+was done by hand. See [Testing](#testing).
 
 ## Uninstall
 
@@ -168,10 +171,38 @@ reference into the module environment as `CALNODE_IMAGE`, which is what the
 systemd unit and configure-module actually run. The label is the single
 source of truth; nothing is stamped at build time.
 
-Running the test suite needs an NS8 test cluster; see the
-[NS8 testing docs](https://github.com/NethServer/ns8-github-actions/blob/v1/README.md#running-tests-locally).
-From this directory: `run-ns8-tests <NODE_ADDR> ghcr.io/calnode/ns8-calnode:edge`.
-Not yet run in CI: no test cluster is wired up.
+## Testing
+
+The suite in `tests/` is Robot Framework. It installs the module on a real NS8
+cluster, configures it, checks the route responds, screenshots the settings UI
+and removes the module again:
+
+| file | what it does |
+|---|---|
+| `10__calnode.robot` | install, configure, route responds |
+| `15__ui.robot` | screenshots of the Status and Settings pages, tagged `ui` |
+| `20__remove.robot` | removal |
+
+Locally, against a cluster you already have (see the
+[NS8 testing docs](https://github.com/NethServer/ns8-github-actions/blob/v1/README.md#running-tests-locally)):
+
+```
+run-ns8-tests <NODE_ADDR> ghcr.io/calnode/ns8-calnode:edge
+```
+
+`NODE_ADDR` must be reachable by SSH as root, using the key at `~/.ssh/id_ecdsa`
+or wherever `SSH_KEYFILE` points. Add `RUN_UI_TESTS=true` to include the `ui`
+tests; without it they are skipped and the runner uses a lighter image.
+
+In CI, `.github/workflows/test-module.yml` does the same thing and provisions
+the cluster on DigitalOcean. It is manual-only for now because it needs a
+`do_token` repository secret this repo does not have yet. It also deliberately
+does not use the stock `test-module.yml@v1`, which derives the image under test
+from the repository name and would resolve it to `ghcr.io/calnode/calnode:latest`
+instead of `ghcr.io/calnode/ns8-calnode:edge`; the workflow header explains the
+full reasoning.
+
+Output lands in `tests/outputs/` and is git-ignored.
 
 ## UI translation
 

packaging/ns8/tests/10__calnode.robot

@@ -0,0 +1,25 @@
+*** Settings ***
+Library    SSHLibrary
+
+*** Variables ***
+${TEST_HOST}    calnode.example.test
+${module_id}    ${EMPTY}
+
+*** Test Cases ***
+Check if ns8-calnode is installed correctly
+    ${output}  ${rc} =    Execute Command    add-module ${IMAGE_URL} 1
+    ...    return_rc=True
+    Should Be Equal As Integers    ${rc}  0
+    &{output} =    Evaluate    ${output}
+    Set Global Variable    ${module_id}    ${output.module_id}
+
+Check if ns8-calnode can be configured
+    ${rc} =    Execute Command    api-cli run module/${module_id}/configure-module --data '{"host": "${TEST_HOST}", "lets_encrypt": false, "http2https": false}'
+    ...    return_rc=True  return_stdout=False
+    Should Be Equal As Integers    ${rc}  0
+
+Check if ns8-calnode works as expected
+    # Host-based route: pass the hostname explicitly since it does not resolve here
+    ${rc} =    Execute Command    curl -f -H 'Host: ${TEST_HOST}' http://127.0.0.1/
+    ...    return_rc=True  return_stdout=False
+    Should Be Equal As Integers    ${rc}  0

packaging/ns8/tests/15__ui.robot

@@ -0,0 +1,37 @@
+*** Settings ***
+Documentation     Settings-UI screenshots. Excluded automatically unless the runner
+...               is started with RUN_UI_TESTS=true.
+
+*** Variables ***
+${ADMIN_USER}    admin
+${ADMIN_PASSWORD}    Nethesis,1234
+
+*** Keywords ***
+Login to cluster-admin
+    New Page    https://${NODE_ADDR}/cluster-admin/
+    Fill Text    text="Username"    ${ADMIN_USER}
+    Click    button >> text="Continue"
+    Fill Text    text="Password"    ${ADMIN_PASSWORD}
+    Click    button >> text="Log in"
+    Wait For Elements State    css=#main-content    visible    timeout=10s
+
+*** Test Cases ***
+Take screenshots
+    [Tags]    ui
+    # Imported here rather than in *** Settings *** on purpose: the runner uses a
+    # different image, without robotframework-browser, whenever RUN_UI_TESTS is
+    # unset. A suite-level import would fail the whole run before the [ui]
+    # exclusion could skip this one test.
+    Import Library    Browser
+    New Browser    chromium    headless=True
+    New Context    ignoreHTTPSErrors=True
+    Login to cluster-admin
+    Go To    https://${NODE_ADDR}/cluster-admin/#/apps/${module_id}
+    Wait For Elements State    iframe >>> h2 >> text="Status"    visible    timeout=10s
+    Sleep    5s
+    Take Screenshot    filename=${OUTPUT DIR}/browser/screenshot/1._Status.png
+    Go To    https://${NODE_ADDR}/cluster-admin/#/apps/${module_id}?page=settings
+    Wait For Elements State    iframe >>> h2 >> text="Settings"    visible    timeout=10s
+    Sleep    5s
+    Take Screenshot    filename=${OUTPUT DIR}/browser/screenshot/2._Settings.png
+    Close Browser

packaging/ns8/tests/20__remove.robot

@@ -0,0 +1,8 @@
+*** Settings ***
+Library    SSHLibrary
+
+*** Test Cases ***
+Check if ns8-calnode is removed correctly
+    ${rc} =    Execute Command    remove-module --no-preserve ${module_id}
+    ...    return_rc=True  return_stdout=False
+    Should Be Equal As Integers    ${rc}  0