Skip to content

cleanup.yml never deletes review images: PAT looks dead and a missing image fails the run #3638

Description

@tom2drum

Summary

The two cleanup_docker_image* jobs in .github/workflows/cleanup.yml
have failed on every run for at least three months. The two cleanup_deployment* jobs in the same
workflow always succeed, so review environments are torn down — but their images are never deleted.

99 of the last 100 runs failed (the 100th was cancelled), spanning 2026-05-062026-08-15, on both
the pull_request and workflow_dispatch triggers.

Evidence

Every failing job ends the same way:

🔎 search package version with tag review-<slug>...
##[error]Package not found.

This is not a case of the image genuinely being absent. In run 31820770464
the job looked for review-plume-testnet-skin, which existed at the time and still exists now:

ghcr.io/blockscout/frontend-private:review-plume-testnet-skin   (version id 1133037416)

Likely root cause

The reusable workflow blockscout/actions/.github/workflows/cleanup_docker.yaml authenticates with a
personal access token:

- name: Delete image
  uses: bots-house/ghcr-delete-image-action@v1.0.0
  with:
    owner: ${{ env.GITHUB_REPOSITORY_OWNER_PART_SLUG }}   # blockscout — correct
    name:  ${{ inputs.imageName ... }}                     # frontend-private — correct
    token: ${{ secrets.PAT }}

owner and name both resolve correctly, which leaves the token. PAT exists as a repo secret in
blockscout/frontend but was last updated 2022-11-29. frontend-private is a private package, and
the GitHub packages API answers an unauthorised or under-scoped request with 404 Package not found
rather than 403 — exactly the error we get. So the token is most likely expired, revoked, or missing
delete:packages.

Worth confirming rather than assuming: it could equally be that secrets: inherit isn't propagating PAT
into the reusable workflow.

Second, independent bug

cleanup_docker_image_2 targets review-2-<slug>. That image only exists if the branch was deployed with
the review-2 variant, which is rare — so for almost every branch it is legitimately absent, the action
errors, and the whole run goes red. Even with a working token, this job would keep failing.

A missing image should be a no-op, not a failure. ghcr-delete-image-action does not appear to offer a
"tolerate missing" switch at v1.0.0, so this probably needs continue-on-error: true on those jobs, or a
newer/different action.

Impact

  • 1680 versions currently exist under frontend-private, of which 84 still carry a review-* tag
    and 1531 are untagged. Nothing has been reclaimed in months.
  • Cleanup runs are permanently red, which trains everyone to ignore them — including for the deployment
    jobs that do matter.

Suggested fix

  1. Rotate PAT with delete:packages + read:packages, and confirm it reaches the reusable workflow.
  2. Make a missing image a no-op so review-2 stops failing runs where only review was deployed.
  3. One-off sweep of the 84 leftover review-* tags and the untagged versions.

Steps 1–2 partly live in blockscout/actions, so this may need a companion PR there.

How to reproduce

gh run list --workflow=cleanup.yml --limit 20 \
  --json event,conclusion,headBranch,createdAt

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions