Skip to content

Add TSG for RegisterCloudManagementClusterExtensions connection refused - #348

Open
bnichms wants to merge 1 commit into
Azure:mainfrom
bnichms:user/bnichms/hcisvc-cloudmanagement-extension-registration
Open

Add TSG for RegisterCloudManagementClusterExtensions connection refused#348
bnichms wants to merge 1 commit into
Azure:mainfrom
bnichms:user/bnichms/hcisvc-cloudmanagement-extension-registration

Conversation

@bnichms

@bnichms bnichms commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

No description provided.

Copilot AI balanced review requested due to automatic review settings September 4, 2026 20:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The guide has diagnostic false positives, unsafe timeout behavior, and missing required metadata and operational safeguards.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Balanced
Findings: 1 Medium severity · 3 Low severity

New issues introduced by this change (4)
Severity Finding
Medium severity TSG/​Update/​Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md — The query returns any 1000/1026 event mentioning HciSvc, but the text declares every result…
Low severity TSG/​Update/​Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md — This new TSG has no hidden metadata marker, so automation cannot identify its document type,…
Low severity TSG/​Update/​Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md — The mitigation starts services, triggers registration sync, and later stops/starts a clustered role…
Low severity TSG/​Update/​README.md — The new inventory entry only adds a link, so it omits the routing and validation fields required…
What changed in this PR

Adds a troubleshooting guide for Azure Local solution-update failures caused by HciSvc connection failures.

Changes:

  • Documents symptoms, validation, cause, mitigation, and verification.
  • Adds the guide to the Update index.
File Description
TSG/​Update/​README.md Links the new guide.
TSG/​Update/​Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md Adds troubleshooting and remediation guidance.
Suppressed comments (1)

TSG/Update/Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md:100

  • If LastConnected never advances, this loop silently expires and the script proceeds to restart Cloud Management even though the key-renewing sync was not confirmed. Track success and terminate on timeout before step 3; expose or increase the deadline for the large-cluster case mentioned above.
$deadline = (Get-Date).AddMinutes(5)
while ((Get-Date) -lt $deadline) {
    Start-Sleep -Seconds 20
    $now = (Get-AzureStackHCI).LastConnected
    if ($now -and (-not $before -or $now -gt $before)) { "Sync completed at $now"; break }

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +62 to +63
Any entries returned confirm this issue. Nothing returned on any node means the update failed for
another reason.
@@ -0,0 +1,139 @@
# Symptoms
Comment on lines +85 to +88
# 1. Start HciSvc wherever it is stopped.
Invoke-Command (Get-ClusterNode) {
if ((Get-Service HciSvc).Status -ne 'Running') { Start-Service HciSvc }
}
Comment thread TSG/Update/README.md
* [Update fails due to Arc Agent install failure (lockdown on azcmagent.log)](./Update-fails-due-to-Arc-Agent-install-failure.md)
* [2604 | Previous CAU run status was canceled](./Previous-CAU-run-status-was-canceled.md)
* [Live migration fails with "No mapping between account names and security IDs" (0x80070534)](./Live-migration-fails-with-No-mapping-between-account-names-and-security-IDs.md)
* [Solution update fails at RegisterCloudManagementClusterExtensions with connection refused on port 42545](./Known-Issue-Solution-update-fails-at-RegisterCloudManagementClusterExtensions-with-connection-refused.md)
@AlBurns-MSFT

Alex Burns (AlBurns-MSFT) commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Ran this on lab hardware on a 2608 build. The mitigation itself works, I took the Cloud Management group offline, confirmed 42545 then refused, ran the steps verbatim, and the port came back. Three things need fixing.

1. The crash check can miss a real instance. StartTime=$boot only searches since the current boot, so if a node restarted after the crash, which is the norm during a solution update, the evidence is filtered out and the operator is told the update failed for another reason. Widening that window, or treating the crash events as a supporting signal rather than the gate, would close it. Separately, when nothing is returned, pointing at TSG/Upgrade/Known-Issue-Cloud-Management-Cluster-Group-Missing-or-Failed-To-Start.md would be more useful than a dead end, since the same symptom appears when the Cloud Management group is offline for other reasons. [LOW RISK]

2. The verification block cannot print what the article says it prints. Invoke-WebRequest throws on a non-2xx in Windows PowerShell 5.1, and .StatusCode is read inside the [pscustomobject] literal, so when it throws the object is never constructed and that node emits no row at all. The block prints an empty table plus one error per node, which is the same thing a broken cluster produces, so the reader cannot tell them apart. That defeats the "if HciSvc were still failing there would be no response at all" line directly below it. On the build I tested, that call returned 401 Unauthorized on both nodes of a healthy cluster, reproduced under elevated admin, SYSTEM, and -UseDefaultCredentials, so the documented 200 table never appeared at all. I only tested one build, so treat the 401 itself as an observation rather than a universal claim, but the code shape is a problem on any build. /metadata/instance with -UseDefaultCredentials does return 200 on the same nodes, so the endpoint is fine, it is this specific call that is not usable. Wrapping it in try/catch and reading $_.Exception.Response.StatusCode, then treating any HTTP response as proof the listener is up, would fix both halves. [MEDIUM RISK]

3. The state-changing steps carry no risk labels and no preconditions. CONTRIBUTING.md puts this under Code Safety (CRITICAL), "All PowerShell/scripts MUST be safe for production", then "Label state-changing commands [LOW RISK], [MEDIUM RISK], or [HIGH RISK]" and "Include a pre-check, expected output, stop condition, rollback, and verification". .github/copilot-instructions.md repeats both, in the PowerShell guidelines ("Assign one canonical risk label") and in the TSG checklist ("Preconditions, workload impact, rollback, and escalation criteria are explicit"). There are three mutating steps here and none of them carries either. It matters more than usual because the reader is by definition on a cluster in the middle of a failed update. Concretely: Start-Service HciSvc fanned out to every node is [LOW RISK], it is guarded by a state check and is reversible; Sync-AzureStackHCI is [MEDIUM RISK], it mutates the Azure-side registration, has no pre-check, and is not undoable; the Stop-ClusterGroup and Start-ClusterGroup pair is [MEDIUM RISK], because it is a stop rather than a move, so cloud management is offline for the duration on every topology including single node, and the pair is not wrapped, so an interrupted session leaves the group down. Nothing states whether any of this is safe to run mid-update, there is no check that the action plan is genuinely failed rather than retrying, no Get-CauRun, and no impact line. The two read-only blocks would take [READ-ONLY], which the guidance says to use as an action type rather than a risk label.

Smaller things:

  • The verification checks 42542 while the failure is on 42545, and those are two different processes. In my repro 42542 answered normally at the same moment 42545 was refusing, so a good result there does not tell you the failure cleared. A Test-NetConnection -Port 42545 against the group owner would close it. [LOW RISK]
  • The comment above the wait loop says to allow longer than five minutes on 16 or more nodes, but the deadline is a literal AddMinutes(5) with no variable to change. The loop also has no timeout branch, so on expiry it falls through to the group restart with no verdict, and if $before comes back null the -not $before test makes the first non-null $now report success after one twenty second sleep. [LOW RISK]
  • No applicability statement. "On earlier versions" has no floor, so a reader cannot tell how far back this applies. .github/copilot-instructions.md asks for applicable products and supported versions to be explicit, and the sibling article uses a foundinbuild front matter line for this. [LOW RISK]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants