Skip to content

test(azure): hub e2e coverage for service-principal and entra-id-groups - #264

Draft
JohannesRudolph wants to merge 1 commit into
mainfrom
claude/hub-e2e-coverage-strategy-1dbb36
Draft

test(azure): hub e2e coverage for service-principal and entra-id-groups#264
JohannesRudolph wants to merge 1 commit into
mainfrom
claude/hub-e2e-coverage-strategy-1dbb36

Conversation

@JohannesRudolph

Copy link
Copy Markdown
Member

Adds e2e/ test roots for azure/service-principal and azure/entra-id-groups, and fixes the wiring bugs that surfaced while getting both integrations to tofu validate.

E2E tests

Both follow the established Azure pattern (modules/azure/{storage-account,resource-group,budget-alert}/e2e): a single test_context grab-bag, the module under test sourced via a relative ../ path, a provider.tf pinning subscription and tenant, and a meshstack_building_block with depends_on = [module.<integration>] so the delete run completes before the backplane's federated identity credentials are destroyed.

azure/service-principal azure/entra-id-groups
BBD target type WORKSPACE_LEVEL TENANT_LEVEL
target_ref meshWorkspace / test_context.workspace meshTenant / fixtures.azure.mesh_tenant_id
per-run backplane name hub-e2e-sp-<suffix> hub-e2e-eig-<suffix[0:12]> (UAMI names cap at 24 chars)
asserted outputs application_id, service_principal_object_id, tenant_id, subscription_id, role_name == "Contributor", secret_expiration_date 3 group object IDs, .admin/.user/.reader display-name suffixes, the per-run prefix

Neither test uses the bbd_version_ref / count dual-mode gating from the e2e skill's checklist: both backplanes declare their own provider blocks, and OpenTofu refuses count on a module whose tree contains provider configurations. This matches what the three existing Azure e2e tests already do for the same reason.

entra-id-groups deliberately passes an empty administrative_unit_id — the fixtures tenant has no Administrative Unit, and creating one per run would leave directory-level litter behind on a failed teardown. The AU membership path is therefore not covered.

Bug fixes

Four pre-existing defects, all of which blocked tofu validate on the integration roots:

  1. service-principal — the azure_subscription_id BBD input used assignment_type = "PLATFORM_TENANT_ID" on a WORKSPACE_LEVEL definition, where there is no platform tenant to resolve it from. Switched to STATIC backed by a new azure_subscription_id variable, matching azure/storage-account and azure/budget-alert.
  2. entra-id-groups — the integration passed scope = var.azure_scope to a backplane module that declares no scope variable. That backplane creates no azurerm role assignments (the UAMI only ever needs Microsoft Graph permissions), so the unused azure_scope variable is dropped rather than plumbed through.
  3. entra-id-groupsPLATFORM_TENANT_WORKSPACE_IDENTIFIER / PLATFORM_TENANT_PROJECT_IDENTIFIER are not valid assignment types. The provider accepts WORKSPACE_IDENTIFIER / PROJECT_IDENTIFIER, which is what ske/ske-starterkit and stackit/git-repository use for the same purpose.
  4. entra-id-groups — a TENANT_LEVEL definition requires spec.supported_platforms; added [{ name = "AZURE" }].

backplane/README.md for entra-id-groups referenced the removed var.scope and described admin consent as a separate portal step. It now names the actual Microsoft Graph app roles required and notes that Application Administrator / Cloud Application Administrator cannot consent to Microsoft Graph application permissions.

Verification

  • tofu validate passes for both e2e/ roots, both backplane/ tiers and the entra-id-groups buildingblock/.
  • tofu fmt -recursive clean; no trailing whitespace.
  • Scorecard Testing 100% for both modules. azure/entra-id-groups is 100% overall; azure/service-principal sits at 74% because its backplane predates the UAMI convention (app registration + create_service_principal_name toggle) — pre-existing and out of scope here.

CI is expected to fail on the Microsoft Graph app-role assignments in both backplanes until the smoke-test CI identity is granted the corresponding directory permissions. That grant is being handled separately; run links and the exact error are in the review thread.

🤖 Generated with Claude Code

Adds e2e/ test roots for both modules following the established Azure pattern
(modules/azure/{storage-account,resource-group}/e2e): a single test_context
grab-bag, the module sourced via a relative path, and a meshstack_building_block
that depends_on the whole integration module so the delete run finishes before
the backplane's federated identity credentials are torn down.

azure/service-principal is WORKSPACE_LEVEL, so the building block targets the
meshWorkspace; azure/entra-id-groups is TENANT_LEVEL and targets the Azure
meshTenant. Both derive a per-run backplane name from name_suffix so custom role
definitions, app registrations and UAMI names don't clash across retried runs.

Also fixes four wiring bugs that blocked both integrations from even reaching
`tofu validate`:

- azure/service-principal: the azure_subscription_id BBD input used
  PLATFORM_TENANT_ID on a WORKSPACE_LEVEL definition, where no platform tenant
  exists. Switched to STATIC backed by a new azure_subscription_id variable,
  matching azure/storage-account and azure/budget-alert.
- azure/entra-id-groups: the integration passed `scope` to a backplane module
  that declares no such variable. The backplane creates no azurerm role
  assignments, so the now-unused azure_scope variable is dropped.
- azure/entra-id-groups: PLATFORM_TENANT_WORKSPACE_IDENTIFIER /
  PLATFORM_TENANT_PROJECT_IDENTIFIER are not valid assignment types; the
  provider accepts WORKSPACE_IDENTIFIER / PROJECT_IDENTIFIER.
- azure/entra-id-groups: a TENANT_LEVEL definition requires
  spec.supported_platforms.

The backplane README's required-permissions table referenced the removed
var.scope and implied a separate portal consent step; it now names the actual
Microsoft Graph app roles and Entra directory role needed to grant them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Scorecard Check

Scorecard run on commit 6ff7e21adb249499a8516798cd2910f6af0a6608 relative to origin/main

📊 meshstack-hub Module Scorecard

Generated: 2026-08-20 | Modules scanned: 2 | Categories: 5

📋 Per-Module Category Summary

Score per category per building block. n/a = category does not apply to this module.

Module Overall Core Structure Integration Azure Backplane STACKIT Backplane Testing
azure/entra-id-groups 🟢 100% 🟢 100% 🟢 100% 🟢 100% n/a 🟢 100%
azure/service-principal 🟡 74% 🟢 100% 🟢 100% 🔴 11% n/a 🟢 100%

⚠️ 1 module has failing checks — failing categories are expanded below.

Core Structure — ✅ all passing

Basic module file structure and documentation — applies to 2 modules

Module Score 📦 🔗 📋 📝 🖼️ 📌 🔒
azure/entra-id-groups 🟢 100%
azure/service-principal 🟢 100%

Core Structure — Summary

Emoji Criterion Coverage Status
📦 buildingblock/ directory exists 2/2 🟢 100%
🔗 meshstack_integration.tf present 2/2 🟢 100%
📋 buildingblock/APP_TEAM_README.md present (no-integration fallback) n/a
📝 buildingblock/README.md with YAML front-matter 2/2 🟢 100%
🖼️ buildingblock/logo.png included 2/2 🟢 100%
📌 buildingblock/versions.tf present 2/2 🟢 100%
🔒 Provider versions use minimum constraint (>=) 2/2 🟢 100%
Integration — ✅ all passing

meshstack_integration.tf conventions — applies to 2 modules

Module Score 🏷️ 🏢 📤 🔌 📎 🔀 📋 🏷️ 🧱 📖 📝 📊 🚫 🔄
azure/entra-id-groups 🟢 100%
azure/service-principal 🟢 100%

Integration — Summary

Emoji Criterion Coverage Status
🏷️ variable "hub" in integration 2/2 🟢 100%
🏢 variable "meshstack" in integration 2/2 🟢 100%
📤 building_block_definition output exposed 2/2 🟢 100%
🔌 meshcloud/meshstack in required_providers 2/2 🟢 100%
📎 backplane source uses var.hub.git_ref 2/2 🟢 100%
🔀 ref_name uses var.hub.git_ref 2/2 🟢 100%
📋 version_spec.draft uses var.hub.bbd_draft 2/2 🟢 100%
🏷️ BBD metadata.tags forwards var.meshstack.tags 2/2 🟢 100%
🧱 BBD input argument vars with optional() have explicit defaults 2/2 🟢 100%
📖 BBD readme field present 2/2 🟢 100%
📝 BBD readme starts with plain-text description (no heading) 2/2 🟢 100%
📊 BBD readme has shared responsibility table (✅/❌) 2/2 🟢 100%
🚫 No documentation_md output in backplane 2/2 🟢 100%
🔄 meshstack_platform has lifecycle ignore_changes = [availability] n/a
Azure Backplane — some checks failing

Azure UAMI-based automation principal conventions — applies to 2 modules

Module Score 🪪 🚫 🚫 🔑 🔗 🧹 📤 📍
azure/entra-id-groups 🟢 100%
azure/service-principal 🔴 11%

Azure Backplane — Summary

Emoji Criterion Coverage Status
🪪 Uses azurerm_user_assigned_identity 1/2 🟡 50%
🚫 No azuread_application resources 1/2 🟡 50%
🚫 No azuread_service_principal resources 1/2 🟡 50%
🔑 No azuread_application_password resources 1/2 🟡 50%
🔗 Uses azurerm_federated_identity_credential 1/2 🟡 50%
workload_identity_federation is non-nullable 2/2 🟢 100%
🧹 No create_service_principal_name toggle 1/2 🟡 50%
📤 Outputs identity (client_id, principal_id, tenant_id) 1/2 🟡 50%
📍 Integration has azure_location 1/2 🟡 50%
STACKIT Backplane — not applicable

STACKIT WIF-based automation principal conventions — applies to 0 modules

No applicable modules.

Testing — ✅ all passing

End-to-end test coverage — applies to 2 modules

Module Score ⚙️ 🧪
azure/entra-id-groups 🟢 100%
azure/service-principal 🟢 100%

Testing — Summary

Emoji Criterion Coverage Status
⚙️ backplane/ directory (optional tier) 2/2 🟢 100%
🧪 e2e/ test directory exists 2/2 🟢 100%
e2e/ contains .tftest.hcl files 2/2 🟢 100%

📈 Overall Summary

Overall Average Score: 87%

Score Distribution

  • 🟢 High maturity (≥80%): 1 modules
  • 🟡 Medium maturity (50–79%): 1 modules
  • 🔴 Low maturity (<50%): 0 modules

@JohannesRudolph

JohannesRudolph commented Aug 20, 2026

Copy link
Copy Markdown
Member Author

CI results — blocked on the expected Entra permission, and nothing else

Both modules were dispatched against this branch in the internal e2e runner. Both fail at the same point, with the same error, before any resource is created:

Error: Listing service principals for filter "appId eq '00000003-0000-0000-c000-000000000000'"

  with module.entra_id_groups.module.backplane.data.azuread_service_principal.msgraph,
  on modules/azure/entra-id-groups/backplane/main.tf line 23, in data "azuread_service_principal" "msgraph":
  23: data "azuread_service_principal" "msgraph" {

unexpected status 403 (403 Forbidden) with error:
Authorization_RequestDenied: Insufficient privileges to complete the
operation.

azure/service-principal is identical, at modules/azure/service-principal/backplane/main.tf line 53.

00000003-0000-0000-c000-000000000000 is Microsoft Graph. The CI identity cannot even read the Microsoft Graph service principal, let alone assign app roles from it — so this fails one step earlier than the app-role assignment itself. Everything upstream of it is green: checkout, provider build, az login, meshStack auth, test context resolution, module source resolution against this branch's SHA, and tofu init of both e2e/ roots including the four integration fixes in this PR.

No errored_test.tfstate was produced for either run — the failure is at plan time, so nothing leaked.

The corresponding Entra directory grant is tracked separately and needs an Entra admin to apply. Until then this PR cannot go green, which is expected and is the only thing outstanding.

@aws-amplify-eu-central-1

Copy link
Copy Markdown

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-264.d1o16zfeoh2slu.amplifyapp.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant