Skip to content

test(github/repository): add hub e2e test coverage - #261

Draft
JohannesRudolph wants to merge 1 commit into
mainfrom
claude/e2e-github-repository
Draft

test(github/repository): add hub e2e test coverage#261
JohannesRudolph wants to merge 1 commit into
mainfrom
claude/e2e-github-repository

Conversation

@JohannesRudolph

Copy link
Copy Markdown
Member

Adds modules/github/repository/e2e/ — the module previously scored 0% on the scorecard's Testing category.

What the test does

  • Build-from-source mode (bbd_version_ref == null): sources ../ to build the BBD, wiring the github object from test_context.fixtures.github.
  • Orders a workspace-level building block (the BBD sets no target_type), so target_ref = { kind = "meshWorkspace", name = <workspace> }.
  • Creates a private repository named smoke-test-github-repository-<name_suffix> so concurrent runs cannot collide.
  • Asserts status.status == "SUCCEEDED" first, then all four real outputs — repo_name, repo_full_name, repo_html_url, repo_git_clone_url — via jsondecode(...).

Cleanup

archive_repo_on_destroy is set to false. The BBD default is true, which would leave one archived repository behind per run — unacceptable for a test that runs on a schedule.

Also

requiresBackplane: false is declared in buildingblock/README.md. This module needs no cloud-side setup: the GitHub App credentials (org, app id, installation id, PEM key) are supplied as static inputs by the platform team.

Scorecard

github/repository Testing: 0% → 100% (overall 🟢 90%).

⚠️ Blocked on a GitHub App permission — needs a decision

The e2e code is complete and correct, but the test cannot pass yet: the GitHub App backing the test fixtures has actions:write, contents:write, metadata:read, workflows:write and is installed on selected repositories. Creating a repository requires org-level administration: write, which the app does not have at all. Deleting the repository on teardown needs the same permission.

Granting org-wide administration: write to a credential used by a scheduled test is a meaningful blast-radius increase (that permission allows renaming, transferring, and deleting any repository in the org), and repository creation in the main meshcloud org would add audit noise, apply org rulesets to throwaway repos, and risk name collisions.

Recommendation: point this test's fixtures at a dedicated throwaway GitHub organization with its own App, rather than widening the existing app's permissions on meshcloud. See the discussion thread for details.

Adds `modules/github/repository/e2e/` so the module is smoke-tested end to
end: the test builds the BBD from source (build-from-source mode), orders a
workspace-level building block that creates a private repository named with
the run's name_suffix, and asserts the run SUCCEEDED plus the four repository
outputs (name, full name, html url, clone url).

`archive_repo_on_destroy` is set to false so teardown deletes the repository
instead of archiving it — a scheduled smoke test must not accumulate archived
repositories in the owning organization.

Also declares `requiresBackplane: false` on the building block: the GitHub App
credentials are supplied as static inputs by the platform team, so there is
nothing cloud-side for a backplane to provision.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Scorecard Check

Scorecard run on commit d17dd027a67bd57f74800c04f42edcefa8e0c58c relative to origin/main

📊 meshstack-hub Module Scorecard

Generated: 2026-08-20 | Modules scanned: 1 | Categories: 5

📋 Per-Module Category Summary

Score per category per building block. n/a = category does not apply to this module.

Module Overall Core Structure Integration Azure Backplane STACKIT Backplane Testing
github/repository 🟢 90% 🟡 67% 🟢 100% n/a n/a 🟢 100%

⚠️ 1 module has failing checks — failing categories are expanded below.

Core Structure — some checks failing

Basic module file structure and documentation — applies to 1 modules

Module Score 📦 🔗 📋 📝 🖼️ 📌 🔒
github/repository 🟡 67%

Core Structure — Summary

Emoji Criterion Coverage Status
📦 buildingblock/ directory exists 1/1 🟢 100%
🔗 meshstack_integration.tf present 1/1 🟢 100%
📋 buildingblock/APP_TEAM_README.md present (no-integration fallback) n/a
📝 buildingblock/README.md with YAML front-matter 1/1 🟢 100%
🖼️ buildingblock/logo.png included 1/1 🟢 100%
📌 buildingblock/versions.tf present 0/1 🔴 0%
🔒 Provider versions use minimum constraint (>=) 0/1 🔴 0%
Integration — ✅ all passing

meshstack_integration.tf conventions — applies to 1 modules

Module Score 🏷️ 🏢 📤 🔌 📎 🔀 📋 🏷️ 🧱 📖 📝 📊 🚫 🔄
github/repository 🟢 100%

Integration — Summary

Emoji Criterion Coverage Status
🏷️ variable "hub" in integration 1/1 🟢 100%
🏢 variable "meshstack" in integration 1/1 🟢 100%
📤 building_block_definition output exposed 1/1 🟢 100%
🔌 meshcloud/meshstack in required_providers 1/1 🟢 100%
📎 backplane source uses var.hub.git_ref 1/1 🟢 100%
🔀 ref_name uses var.hub.git_ref 1/1 🟢 100%
📋 version_spec.draft uses var.hub.bbd_draft 1/1 🟢 100%
🏷️ BBD metadata.tags forwards var.meshstack.tags 1/1 🟢 100%
🧱 BBD input argument vars with optional() have explicit defaults 1/1 🟢 100%
📖 BBD readme field present 1/1 🟢 100%
📝 BBD readme starts with plain-text description (no heading) 1/1 🟢 100%
📊 BBD readme has shared responsibility table (✅/❌) 1/1 🟢 100%
🚫 No documentation_md output in backplane 1/1 🟢 100%
🔄 meshstack_platform has lifecycle ignore_changes = [availability] n/a
Azure Backplane — not applicable

Azure UAMI-based automation principal conventions — applies to 0 modules

No applicable modules.

STACKIT Backplane — not applicable

STACKIT WIF-based automation principal conventions — applies to 0 modules

No applicable modules.

Testing — ✅ all passing

End-to-end test coverage — applies to 1 modules

Module Score ⚙️ 🧪
github/repository 🟢 100%

Testing — Summary

Emoji Criterion Coverage Status
⚙️ backplane/ directory (optional tier) n/a
🧪 e2e/ test directory exists 1/1 🟢 100%
e2e/ contains .tftest.hcl files 1/1 🟢 100%

@aws-amplify-eu-central-1

Copy link
Copy Markdown

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-261.d1o16zfeoh2slu.amplifyapp.com

@JohannesRudolph

Copy link
Copy Markdown
Member Author

CI evidence for the permission blocker

A scheduled-suite run against this branch gets all the way through BBD creation, building block ordering, and into the Terraform apply — then fails on the one API call that needs a permission the fixture App does not have:

Error: POST https://api.github.com/orgs/meshcloud/repos: 403 Resource not
accessible by integration []

  with github_repository.repository,

Everything around it is healthy: the BBD built from this branch, the building block was ordered against the right workspace-level target, and teardown was clean — no errored_test.tfstate was produced, so the delete run and the BBD destroy both completed. The test will go green the moment repository creation is permitted.

What is missing

The App backing the fixture credentials has:

Permission Level
actions write
contents write
metadata read
workflows write

Installation scope: selected repositories.

Creating a repository needs org-level administration: write — the App has no administration permission at any level. Deleting it on teardown (which this test relies on, see archive_repo_on_destroy = false) needs the same permission.

Options

A. Widen the existing fixture App on meshcloud — add administration: write and change the installation to all repositories.
Cheapest, but administration: write org-wide lets that credential rename, transfer, or delete any repository in meshcloud, and the key lives in a secret store plus CI secrets. Throwaway repos would also land in the real org: audit-log noise, org rulesets applied to them, and a repo-count that grows if a teardown ever fails.

B. Dedicated fixture organization (recommended) — a throwaway GitHub org with its own App holding administration: write, installed org-wide there.
administration: write is then scoped to an org that contains nothing of value, so the blast radius is genuinely bounded, and repository churn stays out of meshcloud. Cost is standing up the org, the App, and three more secrets.

Under option B the fixture wiring gains a second credential set (the existing one still serves the workflow-based test, which needs a real repository in meshcloud); this e2e file reads whatever ends up under fixtures.github, so only the fixture side changes.

My recommendation is B. Handing a scheduled, hourly-ish job org-wide administration: write over the main organization is a bad trade for one smoke test, and creating disposable repositories in meshcloud is undesirable independently of the permission question.

Marking this PR draft until that decision is made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant