Repository navigation
test: deploy a real app with the action in CI, report failures to Slack, and move v1 on release - #25
wmadden-electric wants to merge 10 commits into
Conversation
… Slack A new e2e job runs the action itself on e2e/app: it deploys to a new preview environment, checks the outputs and that the service answers for that environment, then destroys it. It runs with the app's pinned prisma and Composer versions and again moved to prisma@latest. Failures on main and on scheduled runs now post to Slack instead of opening an issue nobody watches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
…ncelled runs to Slack Fixes from review: per-step timeouts and a fetch timeout so destroy always gets its time, a sweep of e2e environments earlier runs left behind (one job per matrix entry at a time so it never removes a live one), exact-version checks on the prisma@latest move, the service token only on the steps that need it, no destroy after a skipped-no-credential run, and Slack alerts for cancelled jobs with a message that names the failing job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
|
✅ Gizmo reviewed d0e19db — posted 0 inline comment(s) this pass. Open findings: none Change walkthroughThis pull request adds a real end-to-end deployment test to CI, routes CI failures to Slack, and automates the CI matrix ( README. The Development section's description of the matrix ("including the npm Together with the rest of the PR (the |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Moving v1 was a manual step after each release, so a release could leave @v1 users on the previous version. The workflow never moves v1 back to an older release and can be run by hand with a release tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
There was a problem hiding this comment.
New findings: 🟡 2 minor · trace
Still open from previous reviews: 🟡 2 minor
… the README The sweep refuses to run without this run's environment name, so an empty name can no longer match every e2e environment. Runs of the tag workflow no longer interleave. The README names the right secret for Slack and says prereleases do not move v1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
…release is promoted A shared concurrency group could cancel a waiting main or scheduled e2e job and post a false Slack alert. The group is gone: the sweep now destroys an e2e environment only once the run named in it has finished, and fails only for leftovers older than a day. The tag workflow listens for released, which covers a promoted prerelease, and ignores prerelease tags when deciding whether v1 is ahead. The app name is defined once, the Slack job is named for its purpose, and the README has a manual removal runbook and the full list of runs that report to Slack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
There was a problem hiding this comment.
New findings: 🟡 2 minor · trace
Still open from previous reviews: 🟡 1 minor
…olerate API errors and races The tag workflow now moves each major tag to its newest published release, whichever release started the run, so a run GitHub cancels while it waits loses nothing. The sweep leaves an environment for the next run on a 403, 429 or 5xx from the GitHub API, and only counts an environment as stuck if it still exists, since another sweep may have just removed it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
…ents once in the sweep The tag workflow fetches tags before choosing and chooses only among releases whose tag exists, so a release that outlived its tag cannot stop every major tag from moving. The sweep collects old environments it failed to destroy and lists environments once afterwards, instead of once per failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
When the last release checked had no tag, the loop exited non-zero and bash -e stopped the step silently. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
Two repositories publish prisma under the dev tag: prisma-cli (the CLI the action runs) and the v7 branch of prisma/orm (the Prisma 7 CLI, which has no deploy command). The tag points at whichever published last, so this check has failed on every scheduled run since 2026-11-14 without anything being wrong with the action. The check of each dev build moves to prisma-cli, which can test the exact build it publishes. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Signed-off-by: willbot <w.a.madden+machine@gmail.com> Signed-off-by: Will Madden <madden@prisma.io>
CI now deploys a real app with the action and checks it works:
What the new
e2ejob doesuses: ./) one2e/app, a Composer app with one service. It deploys to a new preview environment for each run.e2e/verify.mjschecks the outputs:outcomeissucceeded,urlis set and appears inurls, andreport-pathnames a file. It then requests the URL until the service answers with its own environment name.destroyoperation of@prisma/composer/control. TheprismaCLI has nodestroycommand, so the action'smode: destroycan't do this yet.e2e-*environments whose GitHub run has finished, which are ones an earlier run could not destroy. The run id is part of each environment's name, so the sweep never touches an environment a running job is still testing. A leftover that won't go away fails the sweep only once it is a day old.It runs twice.
pinneduses the app's lockfile (prisma 8.0.0-rc.33, Composer 0.38.0).latestmoves the app toprisma@latestand the Composer version that release bundles, so a new release is deployed for real every 6 hours.It runs on pull requests from this repository, pushes to
main, the 6-hourly schedule, and manual runs. Pull requests from forks get no secrets, so it skips them. Without thePRISMA_SERVICE_TOKENsecret, the action reportsskipped-no-credentialand the check step fails, so the job can't pass without deploying.Failures go to Slack
A failed or cancelled job on
main, on a scheduled run, on a manual run or on aprisma-releaseddispatch posts to Slack through theSLACK_WEBHOOK_URLsecret. The message names the failing job and links the run. This replaces the job that opened a GitHub issue, which nobody would have seen.v1moves on every release.github/workflows/major-tag.ymlmoves thev1tag to eachv1.x.yrelease when it is published, or when a prerelease is promoted to a release. Until now this was a manual step, which is whyv1was still on v1.10.0 for a while after v1.11.0 was published. Each run movesv1to the newest publishedv1.x.yrelease, whichever release started it, so it never movesv1back and a run GitHub cancels while waiting loses nothing. It can also be run by hand.v1already points at v1.11.0.How I checked it
main.mjslocally one2e/appagainst real Prisma Cloud. It deployed to a preview environment and set every output. The job summary listed the service,verify.mjspassed, and the destroy step removed the environment.CLI.COMMAND_MOVED, so this job would have caught the--stagebreak.The
devtag check is removedCI no longer checks
prisma@dev. Two repositories publishprismaunder that tag: prisma-cli, and thev7branch of prisma/orm, which publishes the Prisma 7 CLI. The tag points at whichever published last, so the check has failed on every scheduled run since 2026-11-14 with nothing wrong in the action (issue #26). The check of each dev build moves to prisma-cli, which can test the exact build it publishes.Setup needed before merging
PRISMA_SERVICE_TOKEN: a service token for the workspace the test deploys into.PRISMA_WORKSPACE_ID: that workspace's id.The app deploys as project
cloud-deploy-action-e2ein regionus-east-1.Not covered
npm ciand theprisma-versioninput. The test app installs with Bun and carries its ownprismadevDependency.prisma branch list, because@prisma/composer/controlhas no list operation, and destroys them with itsdestroyoperation. Both can move to the action once the CLI has a destroy command.Agent: bombadil-28, taliesin-31
🤖 Generated with Claude Code