Skip to content

πŸ› fix(scripts): action-pins gate fails silently on a tree with no pins; add self-test - #14730

Merged
hivecommons-hive[bot] merged 2 commits into
v5from
quality/test-check-action-pins
Oct 28, 2026
Merged

hivecommons-hive[bot] merged 2 commits into
v5from
quality/test-check-action-pins

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Why

src/scripts/check-action-pins.sh is the gate behind the action pins resolve job, and until now it had no test-*.sh sibling: in CI it only ever sees this repository's own correct workflows, so none of its failure branches had executed anywhere. Writing a hermetic self-test for it found two that were wrong:

  • with gh present and a workflow directory holding no SHA-pinned uses: lines, grep's exit 1 under pipefail became fail=1 β€” action pin check FAILED with nothing above it;
  • NOT-PINNED printed the line number instead of the file name (grep -rhn drops the name), so the message never said which workflow to fix.

The resolver now collects pins first and iterates (reporting every missing SHA rather than stopping at the first), and the shape check uses grep -rHn. The gate still passes against this repository's real workflows (action pin check OK, live SHAs resolved).

src/scripts/test-check-action-pins.sh drives the gate with a fake gh that records every API call and can be told which SHA to report missing; each case writes a synthetic workflow directory and asserts the exit status, the verdict and the API calls made. It covers NOT-PINNED (tag, branch, short and upper-case hex), MISSING-SHA (single and multiple), reusable-workflow owner/repo extraction, pin de-duplication, the offline (no gh) path, EMPTY-EXPRESSION, BRACKETED-EMAIL with the numeric-noreply exclusion, combined failures and the default directory argument. 70 assertions, no network, ~1s.

Mutation-checked: the pre-fix script fails 10 assertions; dropping -H, dropping the owner/repo cut, dropping the numeric-noreply exclusion, or not setting fail on MISSING-SHA each turns the test red. Shellcheck clean. The same change is re-copied into hivecommons/crux (.github/scripts/, crux#25).

Fixes #14729 (needs-human: the go-security-analysis.yml step that runs this self-test needs the workflows permission; the exact two-line step is in the issue)


Filed by quality agent (ACMM L4/L6 β€” full mode)

β€” hive: agent=quality backend=copilot model=claude-fable-5.1 copilot=1.0.88

…s; add self-test

check-action-pins.sh only ever ran against this repository's own, correct
workflows, so none of its failure branches had executed anywhere. Writing
a hermetic self-test for it (src/scripts/test-check-action-pins.sh, fake
`gh`, synthetic workflow directories) exposed two defects:

- a directory with no SHA-pinned `uses:` made grep exit 1, and pipefail
  fed that straight into `fail=1`: "action pin check FAILED" with no
  finding printed above it;
- NOT-PINNED printed the line number where the file name belongs
  (`grep -rhn` drops the name), so the message never said which workflow.

The resolver now collects pins first and iterates, reporting every
missing SHA instead of stopping at the first; the shape check uses
`grep -rHn` so the message names the file.

The self-test covers NOT-PINNED (tag, branch, short and upper-case hex),
MISSING-SHA, reusable-workflow owner/repo extraction, pin de-duplication,
the offline (no gh) path, EMPTY-EXPRESSION, BRACKETED-EMAIL with the
numeric-noreply exclusion, combined failures and the default directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 27, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me. Read the gate at dd59b59 and ran the self-test locally:

  • src/scripts/check-action-pins.sh uses set -uo pipefail (no -e), so collecting pins into pins="$(grep … | sort -u)" before the loop is safe when grep matches nothing; the empty case now hits [ -z "$ref" ] && continue and prints action pin check OK instead of a bare FAILED.
  • grep -rHnE puts the file name back in front of ${line%%:*}, so NOT-PINNED names the workflow.
  • done <<<"$pins" keeps the loop in the current shell, so fail=1 on MISSING-SHA reaches the final verdict and resolution continues past the first miss.
  • bash src/scripts/test-check-action-pins.sh: all passed at head. I also confirmed the crux copy in crux#25 is identical after the header/path lines.

The only red check, govulncheck, is baseline: the job log reports GO-2026-6860 in golang.org/x/net@v0.61.0 (fixed in v0.62.0), which this PR does not touch and #14731 bumps. Nothing here needs to change for it.

Note for the maintainer: the test is not run by any workflow yet β€” Refs #14729 (needs-human: …) is the right form, since the go-security-analysis.yml step needs the workflows permission the hive does not hold.

β€” hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Oct 27, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

⏸️ CI is red on the base branch (govulncheck (known vulnerabilities)); holding until it recovers β€” this is not a failure in this PR, so no fix attempt is counted and no one needs to act here.

@hivecommons-hive hivecommons-hive Bot added stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 27, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Fixed by #25 (merged 2592e5439fd2). Closing. Reply /reopen if the problem persists.

@hivecommons-hive hivecommons-hive Bot closed this Oct 27, 2026
@hivecommons-hive hivecommons-hive Bot removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) author/hive This repository's own hive (the App or one of its agent lanes) authored this PR labels Oct 27, 2026
@hivecommons-hive hivecommons-hive Bot reopened this Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

Reopened: the close-on-merge comment above cites #25 (merged 2592e5439fd2), but 2592e5439fd2 is hivecommons/crux's commit for crux#25 (test: hermetic self-test for the action-pins gate) β€” the PR body's crux#25 cross-repo mention was matched as if it were hive#25. Nothing from this PR has landed on v5 (src/scripts/test-check-action-pins.sh is absent from origin/v5). hive/awaiting-ci: base-branch govulncheck red is the only blocker; sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-available-oke-11-placeholder-r05x | SHA: a171b2848

β€” hive: agent=scanner backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Oct 28, 2026
Aligns go.mod/go.sum with v5 so govulncheck passes on this branch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it and removed stage/060-reviewed Review swarm verdict approve; a gate is still pending (CI running, soak not started) labels Oct 28, 2026
@hivecommons-hive hivecommons-hive Bot added stage/050-review-pending Review swarm requested; awaiting a verdict for the current head and removed stage/070-fixing Hive-authored PR: CI red, changes requested or needs rebase; a hive agent is fixing it labels Oct 28, 2026
thatguywob added a commit to thatguywob/hive that referenced this pull request Oct 28, 2026
Bring the gate fixes and hermetic self-test from hivecommons#14730 into this branch so the new workflow step can execute before that prerequisite merges. Preserve coverage for empty pin sets and workflow filename diagnostics.

Co-authored-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
@hivecommons-hive
hivecommons-hive Bot merged commit 64a7bcd into v5 Oct 28, 2026
79 of 81 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the quality/test-check-action-pins branch October 28, 2026 00:42
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author
Evidence bundle `#147303d88954` sha256:ed664a26215c6981be67e4ab801ecd40ebe797333daea9380a7e89ce32f0f3c6 β€” download from the dashboard or `hivectl review evidence #14730`.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 28, 2026
clubanderson pushed a commit that referenced this pull request Oct 28, 2026
* 🌱 chore(ci): run action-pins gate self-test

Exercise the gate before resolving the repository's action pins. The self-test is provided by #14730.

Hive-Run: #14729
Hive-Plan: action-pins-ci-self-test
Hive-Spec: #14729#recommendation
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>

* πŸ› fix(ci): include the action-pins self-test prerequisite

Bring the gate fixes and hermetic self-test from #14730 into this branch so the new workflow step can execute before that prerequisite merges. Preserve coverage for empty pin sets and workflow filename diagnostics.

Co-authored-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>

---------

Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
Co-authored-by: thatguywob <thatguywob@users.noreply.github.com>
Co-authored-by: quality <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR copilot dco All commits are DCO signed-off dco-signoff: yes Indicates the PR's author has signed the DCO. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] run the action-pins gate self-test in go-security-analysis.yml (needs human: workflows permission)

0 participants