Repository navigation
ci: run CLI checks on disposable ci-1 workers #84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| self-hosted-runner: | ||
| labels: | ||
| - ci1-integration |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: | ||
| - main | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| verify: | ||
| name: Verify Node ${{ matrix.node-version }} | ||
| runs-on: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository && 'ubuntu-latest' || 'ci1-integration' }} | ||
| timeout-minutes: 20 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| node-version: | ||
| - 22 | ||
| steps: | ||
| - name: Require trusted source | ||
| env: | ||
| EVENT_NAME: ${{ github.event_name }} | ||
| HEAD_REPOSITORY: ${{ github.event.pull_request.head.repo.full_name }} | ||
| REPOSITORY: ${{ github.repository }} | ||
| run: | | ||
| set -euo pipefail | ||
| if [[ "$EVENT_NAME" == pull_request && "$HEAD_REPOSITORY" != "$REPOSITORY" ]]; then | ||
| echo 'Fork PRs require a separately qualified runner policy' >&2 | ||
| exit 1 | ||
| fi | ||
| - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Require qualified ci-1 image | ||
| run: | | ||
| python3 scripts/verify-ci1-image.py | ||
| sha256sum --check --strict /etc/ci1-image/manifests.sha256 | ||
| - name: Test image admission refusals | ||
| run: python3 -m unittest discover -s scripts -p ci1_image_test.py -v | ||
| - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| cache: npm | ||
| - run: npm ci | ||
| - run: npm run verify | ||
| - run: node --test scripts/release-evidence.test.mjs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # CI migration checkpoint | ||
|
|
||
| Collaborator: purpshell. Agent/model: Codex/gpt-6.1-sol. Host: T3 Code. | ||
| Thread: `8ad9e447-6fd5-4fc5-8e96-189326af2c28`. | ||
| Source dependency: CLI main `c21af71f13e68230769d1a2ca17985d143bf6c6e`. | ||
| Registration: https://github.com/polymorfa/polymorfa/pull/511. | ||
|
|
||
| Coverage: manually summarized visible messages and selected tool outcomes. | ||
| This is not automatic T3 capture. Excludes raw logs, private credential paths or | ||
| values, attachments, authentication screens, unrelated history and hidden model | ||
| reasoning. Checkpoint date is 2026-10-19; original message arrival times are not | ||
| reconstructed. Attribution and review are self-reported. | ||
|
|
||
| ## Visible instructions and outcomes | ||
|
|
||
| - User approved GARM + Incus and versioned Ubuntu images under the coordinated | ||
| monorepo plan, and chose to pause new paid CI while required checks wait. | ||
| - User then asked to get the cutover going and get it out as soon as possible. | ||
| - GitHub confirms this repository is private and old CI workflow 338004193 is | ||
| disabled. Its existing source uses Blacksmith with Node 22 and three commands. | ||
| - Agent created an isolated CLI worktree from the fetched main SHA above. | ||
| No AGENTS.md or CLAUDE.md exists in that source; README and both workflows | ||
| were inspected. The old checkout remains on its original branch. | ||
| - The prepared workflow uses ci1-integration and retains Node 22 and the three | ||
| existing source commands. New image admission rejects stale or mismatched | ||
| definitions and verifies the worker shape. Five local boundary tests pass; | ||
| actionlint and zizmor pass after declaring the actual custom runner label. | ||
| - The new workflow has read-only permissions, pinned checkout/Node actions, | ||
| refuses fork PRs and uses a 20-minute bound. There is no paid fallback. | ||
| - Runtime code, public contracts, package publication and deployment are | ||
| unchanged. Actual GitHub CI acceptance remains pending at this checkpoint. | ||
|
|
||
| - Full local verification passed 110 test files and 1,051 tests, plus types, lint, | ||
| formatting, build, package dry-run and a clean production audit. The three | ||
| release-evidence tests also passed. No package or lockfile changed. | ||
|
|
||
| ## Review follow-up | ||
|
|
||
| The user asked to handle the three findings and merge urgently. Two findings | ||
| reproduced: a skipped fork job can satisfy the required check, and default GNU | ||
| checksum parsing accepts a malformed row beside a valid row. The fix fails the | ||
| same named check before checkout on a GitHub-hosted fork rejection worker and | ||
| uses strict checksum parsing. All trusted source work remains on ci-1. Seven | ||
| boundary tests pass after reproducing both failures. | ||
|
|
||
| The final-image identity is already bound at the controller: live readback | ||
| confirmed scale sets 1 and 2 pinned to the qualified fingerprint, with its | ||
| fingerprint-named Incus alias pointing to itself. The provider rejects remote | ||
| names and retargeted aliases. The new operator receipt documents that authority; | ||
| the guest's manifest is a content/age check rather than image attestation. | ||
| No pool, image, host limit, product secret, deployment or runtime changed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| { | ||
| "schema": 1, | ||
| "observed_at": "2026-10-19T19:00:23.666736+00:00", | ||
| "controller": "ci-1", | ||
| "scalesets": [ | ||
| { | ||
| "id": 1, | ||
| "name": "ci1-general", | ||
| "image": "505d3336ea46318175907ac2192df5002406731bd2990a344ba2ee98c1c89ab0", | ||
| "enabled": true | ||
| }, | ||
| { | ||
| "id": 2, | ||
| "name": "ci1-integration", | ||
| "image": "505d3336ea46318175907ac2192df5002406731bd2990a344ba2ee98c1c89ab0", | ||
| "enabled": true | ||
| } | ||
| ], | ||
| "incus_project": "ci1", | ||
| "alias": { | ||
| "name": "505d3336ea46318175907ac2192df5002406731bd2990a344ba2ee98c1c89ab0", | ||
| "target": "505d3336ea46318175907ac2192df5002406731bd2990a344ba2ee98c1c89ab0" | ||
| }, | ||
| "authority": "GARM controller scale-set image pin and provider fingerprint-alias enforcement; guest manifest is a separate content and age check", | ||
| "mutation": "none; read-only verification", | ||
| "requalification": "Any new fingerprint needs explicit pool changes and the coordinated image/lifecycle qualification", | ||
| "provider_wrapper": { | ||
| "path": "/opt/garm/providers.d/ci1-incus", | ||
| "sha256": "a091d8c0f8a08be521d754d046622b6fcc66f127c9bea52d60082f2a3d424813", | ||
| "matches_source": "https://github.com/polymorfa/polymorfa/blob/d35de858d73fe9c6d9e6110637a18a3dfe57fb56/infra/ci1/provider-admission.sh", | ||
| "checks": [ | ||
| "local immutable fingerprint required", | ||
| "fingerprint-named alias must target that exact fingerprint", | ||
| "remote names rejected" | ||
| ] | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| # CLI checks on ci-1 | ||
|
|
||
| The owner approved the GARM + Incus cutover and paused new paid CI runs. This | ||
| change qualifies the private CLI repository on disposable Linux workers. The | ||
| [coordinated plan](https://github.com/polymorfa/polymorfa/blob/d35de858d73fe9c6d9e6110637a18a3dfe57fb56/internal/docs/superpowers/plans/2026-10-19-ci1-runner-images-and-cutover.md) | ||
| owns the host, image, lifecycle proof and environment holds. | ||
|
|
||
| ## Workflow and image contract | ||
|
|
||
| `ci1.yml` keeps the CI display name, Verify Node 22 check and all three existing | ||
| commands: `npm ci`, `npm run verify` and the release-evidence tests. It uses | ||
| `ci1-integration` with eight CPUs and 32 GiB RAM, a 20-minute job deadline, read-only | ||
| repository permission and no checkout credential persistence. The already qualified integration pool lets this CLI check run alongside the | ||
| monorepo general queue, within the unchanged host budget. Fork PRs run only a | ||
| failing guard on a GitHub-hosted runner, before checkout; the preserved Verify | ||
| Node 22 check fails instead of reporting a successful skip. Trusted source work | ||
| always uses ci-1. No source-test fallback exists. | ||
|
|
||
| The new file identity lets this PR qualify while old paid workflow 338004193 | ||
| stays disabled. Do not enable the old identity before inspecting its source; | ||
| the default branch still selects Blacksmith until the cutover merges. | ||
|
|
||
| Before installation, the job validates the shared image's base fingerprint, | ||
| machine/tool input hashes, age of at most seven days, architecture and CPU shape. | ||
| It then verifies the image's checksum manifest with strict parsing; any malformed | ||
| line fails admission. Node setup selects cached Node 22. Existing npm cache behavior and the lockfile-based installation remain. | ||
| The image's pnpm cache belongs to the monorepo; no CLI npm-cache hit is inferred. | ||
| A changed image definition requires new qualification and explicit pin updates. | ||
|
|
||
| The qualified image fingerprint is | ||
| `505d3336ea46318175907ac2192df5002406731bd2990a344ba2ee98c1c89ab0`. | ||
| The controller's scale set 2 (`ci1-integration`) pins this exact final fingerprint; | ||
| scale set 1 does too. A live readback on 2026-10-19 confirmed both pins and the | ||
| fingerprint-named local Incus alias targeting that same fingerprint. The installed | ||
| provider wrapper SHA-256 also matches its committed source. The | ||
| [readback receipt](ci1-qualified-pool.json) records the binding. The controller | ||
| provider refuses remote or non-fingerprint image names and rejects retargeted | ||
| aliases. Image identity is bound by this controller configuration; the guest | ||
| manifest checks content and age. A new image needs explicit operator pool | ||
| changes and full qualification under the coordinated plan. | ||
|
|
||
| Its definition source is `6481b9e199d96fb3488f34d6335ac41169f4fb5d` in the monorepo. | ||
| The underlying Ubuntu base is | ||
| `2139edd9da4026da8a66c0557af7eee4789101104eb27860bacd57086219cebb`. | ||
| Public source dependencies only are installed; no customer credential, release | ||
| signing key or production secret is added to CI or the image. | ||
|
|
||
| ## Qualification and rollout | ||
|
|
||
| Source dependency: CLI main `c21af71f13e68230769d1a2ca17985d143bf6c6e`. | ||
| Seven local admission tests cover the accepted image, wrong architecture/shape, | ||
| expiry/future clock, wrong base, changed/missing definition or tool hashes, | ||
| fork guard outcomes before checkout and malformed checksum rows. The two new | ||
| regressions failed before the fix and pass afterward; all source test commands | ||
| remain unchanged. | ||
| The unchanged local `npm run verify` passed all 110 test files and 1,051 tests, | ||
| types, lint, formatting, build, package dry-run and production audit (zero | ||
| production vulnerabilities). Three release-evidence tests also passed. | ||
| Local workflow lint and diff checks pass. GitHub source qualification and the | ||
| first main-branch run remain required. Host lifecycle proof is separate from CLI | ||
| source acceptance. | ||
|
|
||
| The publication workflow is unchanged. It runs on its schedule or manual | ||
| invocation, requires matching SDK/API acceptance and never runs merely because | ||
| this CI PR merges. This change does not authorize a package publication or | ||
| staging deployment. Staging and regional rename holds remain in force. | ||
|
|
||
| On a bad image, drain/rebuild the worker pool under the monorepo runbook and | ||
| qualify it again. Do not silently re-enable paid CI. Reverting the source change | ||
| does not re-enable workflow 338004193; paid resumption needs an owner cost decision. | ||
|
|
||
| ## Impact and session provenance | ||
|
|
||
| CI and operator documentation are updated. CLI commands, credential handling, | ||
| SDK dependency, API contracts, public docs/changelog, product release flags, | ||
| Admin controls, billing, Usage, logs, analytics and customer interfaces are | ||
| unaffected because no runtime, dependency or product configuration changes. | ||
| No customer feature or release flag is introduced. Linear is unavailable in this | ||
| session; no tracker synchronization is claimed. | ||
|
|
||
| Collaborator purpshell; Codex/gpt-6.1-sol; T3 thread | ||
| `8ad9e447-6fd5-4fc5-8e96-189326af2c28`. The existing registration is | ||
| [monorepo PR 511](https://github.com/polymorfa/polymorfa/pull/511). | ||
| The [sanitized manual checkpoint](../agent-sessions/ci1-cli-20261019.md) records | ||
| visible instructions and tool outcomes, with capture gaps. It provides | ||
| provenance, not GitHub acceptance or release authority. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| """Fail-closed admission for the shared, pinned ci-1 image.""" | ||
| import copy | ||
| import datetime | ||
| import importlib.util | ||
| import pathlib | ||
| import hashlib | ||
| import os | ||
| import subprocess | ||
| import tempfile | ||
| import textwrap | ||
| import unittest | ||
|
|
||
| spec = importlib.util.spec_from_file_location("ci1_image", pathlib.Path(__file__).with_name("verify-ci1-image.py")) | ||
| image = importlib.util.module_from_spec(spec) | ||
| spec.loader.exec_module(image) | ||
|
|
||
|
|
||
| class AdmissionTest(unittest.TestCase): | ||
| def setUp(self): | ||
| self.now = datetime.datetime(2026, 10, 19, 18, 0, tzinfo=datetime.timezone.utc) | ||
| self.manifest = { | ||
| "built_at": "2026-10-19T14:41:57+00:00", | ||
| "base_fingerprint": "2139edd9da4026da8a66c0557af7eee4789101104eb27860bacd57086219cebb", | ||
| "inputs": { | ||
| "infra/ci1/machine.json": "0900619bac06c6a81f5d7535107985bf8d9cf11c6dd2079273d8ab81d9ebac3a", | ||
| "infra/ci1/tools.lock.json": "4c2a7cac77ebd8221054b2e78dfdd81e15327fb92d3e80d8986a73fb5e326d15", | ||
| }, | ||
| } | ||
|
|
||
| def test_qualified_image(self): | ||
| image.validate_image(self.manifest, self.now, "x86_64", 8) | ||
|
|
||
| def test_wrong_shape_or_architecture(self): | ||
| for arch, cpus in [("aarch64", 8), ("x86_64", 4)]: | ||
| with self.subTest(arch=arch, cpus=cpus), self.assertRaises(ValueError): | ||
| image.validate_image(self.manifest, self.now, arch, cpus) | ||
|
|
||
| def test_expired_image_or_future_clock(self): | ||
| for built in ["2026-10-11T18:00:00+00:00", "2026-10-19T18:02:00+00:00", "2026-10-19T14:41:57"]: | ||
| with self.subTest(built=built), self.assertRaises(ValueError): | ||
| image.validate_image({**self.manifest, "built_at": built}, self.now, "x86_64", 8) | ||
|
|
||
| def test_wrong_base(self): | ||
| with self.assertRaises(ValueError): | ||
| image.validate_image({**self.manifest, "base_fingerprint": "0" * 64}, self.now, "x86_64", 8) | ||
|
|
||
| def test_changed_or_missing_definition_and_tools(self): | ||
| for name in self.manifest["inputs"]: | ||
| for value in [None, "0" * 64]: | ||
| bad = copy.deepcopy(self.manifest) | ||
| if value is None: | ||
| del bad["inputs"][name] | ||
| else: | ||
| bad["inputs"][name] = value | ||
| with self.subTest(name=name, value=value), self.assertRaises(ValueError): | ||
| image.validate_image(bad, self.now, "x86_64", 8) | ||
|
|
||
|
|
||
| class WorkflowAdmissionTest(unittest.TestCase): | ||
| def setUp(self): | ||
| self.workflow = pathlib.Path(__file__).parents[1].joinpath(".github/workflows/ci1.yml").read_text() | ||
| self.scratch = pathlib.Path.home() / ".polymorfa-agent-work" / "cli-ci1-admission-tests" | ||
| self.scratch.mkdir(parents=True, exist_ok=True) | ||
|
|
||
| def test_malformed_checksum_row_fails(self): | ||
| with tempfile.TemporaryDirectory(dir=self.scratch) as directory: | ||
| root = pathlib.Path(directory) | ||
| data = root / "protected-file" | ||
| data.write_bytes(b"qualified bytes") | ||
| record = root / "manifest.sha256" | ||
| record.write_text(hashlib.sha256(data.read_bytes()).hexdigest() + " " + str(data) + "\n") | ||
| command = next(line.strip().split() for line in self.workflow.splitlines() if line.strip().startswith("sha256sum --check")) | ||
| command[-1] = str(record) | ||
| result = subprocess.run(command, capture_output=True) | ||
| self.assertEqual(result.returncode, 0, result.stderr.decode()) | ||
| record.write_text(record.read_text() + "malformed entry\n") | ||
| result = subprocess.run(command, capture_output=True) | ||
| self.assertNotEqual(result.returncode, 0, result.stdout.decode()) | ||
|
|
||
| def test_fork_guard_fails_before_checkout(self): | ||
| marker = " - name: Require trusted source\n" | ||
| self.assertIn(marker, self.workflow) | ||
| script = self.workflow.split(marker, 1)[1].split(" run: |\n", 1)[1].split("\n - ", 1)[0] | ||
| script = textwrap.dedent(script) | ||
| for event, head, expected in [("pull_request", "polymorfa/cli", 0), ("push", "", 0), ("pull_request", "external/cli", 1), ("pull_request", "", 1)]: | ||
| with self.subTest(event=event, head=head): | ||
| result = subprocess.run(["bash", "-c", script], env={**os.environ, "EVENT_NAME": event, "HEAD_REPOSITORY": head, "REPOSITORY": "polymorfa/cli"}, capture_output=True) | ||
| self.assertEqual(result.returncode, expected) | ||
| self.assertLess(self.workflow.index(marker), self.workflow.index(" - uses: actions/checkout@")) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| #!/usr/bin/env python3 | ||
| """Validate the shared ci-1 image before installing CLI dependencies.""" | ||
| import datetime | ||
| import json | ||
| import os | ||
| import pathlib | ||
| import platform | ||
|
|
||
| BASE = "2139edd9da4026da8a66c0557af7eee4789101104eb27860bacd57086219cebb" | ||
| INPUTS = { | ||
| "infra/ci1/machine.json": "0900619bac06c6a81f5d7535107985bf8d9cf11c6dd2079273d8ab81d9ebac3a", | ||
| "infra/ci1/tools.lock.json": "4c2a7cac77ebd8221054b2e78dfdd81e15327fb92d3e80d8986a73fb5e326d15", | ||
| } | ||
|
|
||
|
|
||
| def validate_image(data, now, architecture, cpus): | ||
| if architecture != "x86_64" or cpus != 8: | ||
| raise ValueError("ci-1 requires the qualified 8-CPU x86_64 integration VM") | ||
| built = datetime.datetime.fromisoformat(data["built_at"]) | ||
| if built.tzinfo is None: | ||
| raise ValueError("ci-1 image timestamp requires a timezone") | ||
| age = now - built | ||
| if age.total_seconds() < -60 or age > datetime.timedelta(days=7): | ||
| raise ValueError("ci-1 image age or clock is invalid; rebuild before running CI") | ||
| if data["base_fingerprint"] != BASE: | ||
| raise ValueError("ci-1 base image does not match the qualified definition") | ||
| for name, expected in INPUTS.items(): | ||
| if data["inputs"].get(name) != expected: | ||
| raise ValueError("ci-1 image definition or tools changed; qualify before running CI") | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| manifest = pathlib.Path("/etc/ci1-image/input-manifest.json") | ||
| data = json.loads(manifest.read_text()) | ||
| validate_image(data, datetime.datetime.now(datetime.timezone.utc), platform.machine(), len(os.sched_getaffinity(0))) | ||
| print("ci-1 image age, base, tool definition and integration VM shape qualified") | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.