Skip to content

feat(prisma-cloud)!: a service’s COMPOSER_ keys are sent with its deployment, and a deploy writes no variable rows - #451

Draft
wmadden-electric wants to merge 15 commits into
mainfrom
feat/composer-keys-travel-with-the-deployment
Draft

wmadden-electric wants to merge 15 commits into
mainfrom
feat/composer-keys-travel-with-the-deployment

Conversation

@wmadden-electric

Copy link
Copy Markdown
Contributor

A deploy of a service that reads a database and exposes an RPC interface sends one create-deployment request for it, with the service's own configuration in the body:

// POST /v1/services/{serviceId}/deployments
{
  "portMapping": { "http": 3000 },
  "variables": {
    "COMPOSER_SHOP_AUTH_SERVICE_PORT": "3000",
    "COMPOSER_SHOP_AUTH_SERVICE_ORIGIN": "…",
    "COMPOSER_SHOP_AUTH_SERVICE_DB_URL": "…",
    "COMPOSER_SHOP_AUTH_SERVICE_RPC_ACCEPTED_KEYS": "…",
    "COMPOSER_SHOP_AUTH_SERVICE_STREAMS_API_KEY": null
  }
}

It makes no POST or PATCH to /v1/environment-variables. Before this change the same deploy wrote one variable row per key, as a branch override on a preview environment and as a row in the production class on production.

This is a draft, and three of its checks are red on purpose. See "Before this can merge".

What changes

  1. A service's COMPOSER_ keys travel with its deployment. They go in the variables field of the create-deployment request, where the platform stores them on that service. Keys keep their names, so nothing changes in how a service boots.
  2. Every key a service reads at boot is sent on every deploy, as a value or as null. A key that is left out keeps its stored value, so a key the service should no longer have must be deleted explicitly. A streams service that loses its last consumer is deployed with its key as null and refuses to boot, as it does today.
  3. Keys a service used to send are deleted. A new resource in deploy state, PrismaCloud.SentVariableNames, keeps for each service every key name Composer has sent. A name no longer sent goes out as null. It holds names, never values.
  4. More than 50 values for one service is refused before any resource is created, changed or deleted, with the service, the count and the limit.
  5. Under prisma dev, a service that runs its artifact takes its keys from its deployment's variables. A service that runs a dev server has no deployment, so it keeps reading rows from env.json.
  6. The key-clash check is deleted. It existed because two rows could hold one key.

For Ty: the interface this is built against

  • The field is variables on the create-deployment request: key to string, or null to delete.
  • "Keys not sent keep their value" is what we want. Composer manages the keys it knows, and anything else a user set on the service stays theirs. We do not want a way to replace a service's full set: it would delete values a user set in Console on every deploy.
  • We rely on null for a key that is not stored being accepted and doing nothing. Composer sends one for every optional value that is not set, on every deploy.
  • If your final field differs, tell us here and we change patches/@alchemy.run__prisma@…, which is the only place that builds the request.
  • There may be two minimum Composer versions. 0.40.0 is the minimum for projects with a custom variable group. The release that carries this change is the minimum for the day branch override writes are rejected. If this is ready in time it goes into 0.40.0 and there is one number.

What an existing app sees on its first deploy with this version

  1. Each service's deployment is replaced once, created with its variables.
  2. The old variable rows are deleted afterwards, each after the old deployment it fed has stopped. On production, Composer's keys leave the user's production group.
  3. A row already deleted in Console is skipped. A row the platform refuses to delete because its recorded identity does not match is dropped from deploy state with a warning naming the key.
  4. After this deploy, an older Composer cannot deploy that environment: it stops while planning, because deploy state names a resource it has no code for.

Before this can merge

  1. The platform accepts variables in production. Until then Deploy, verify, destroy is red: the request schema rejects unknown fields.
  2. @alchemy.run/prisma and @distilled.cloud/prisma release the field. Until then both carry it as pnpm patches under patches/. Patches are not published with Composer, and the unpatched provider ignores variables: a release from this branch would deploy every service without its database URL, port and keys, and report success. So this pull request adds a check, pnpm check:patched-dependencies, that fails while any run-time dependency of a published package is patched. It fails here today, in Lint and in Publish preview, and passes once Composer depends on the real releases. The proposed upstream changes are in .drive/projects/variable-groups-compat/slices/upstream/.
  3. It merges after the preflight pull request (fix(prisma-cloud): the deploy checks platform variables against the variable group the branch receives #450), whose glossary entries this one shares.
  4. One QA run against the real platform: a fresh deploy, and an upgrade of an app deployed with 0.39.0, on production and a preview.

How it was checked

  • Tests run the real Alchemy engine against a fake platform that records each create request: an unchanged deploy creates no deployment; one changed value replaces only that service; a removed key is sent as null on that deploy and the next; the upgrade from state written by today's code; an interrupted upgrade; a mismatched row. Each test was seen to fail with its behaviour removed.
  • Design review, and two rounds of system-design and code review.
  • Manual QA, two rounds, on real deploys of a two-service app with a database, through a proxy that turned variables into rows so the real platform could run the services. Fresh production and preview deploys wrote no rows and every service answered; an unchanged redeploy created no deployment; removing a generated value redeployed only its service; an app deployed with the published 0.39.0 was upgraded with each deployment replaced once and all 22 old rows deleted after the old deployments stopped; services were watched for 12 minutes afterwards and kept answering; deleting a row a running service needs did not affect it. What this cannot show is that services run on stored service variables; item 4 above covers that.
  • Six Postgres-backed test files could not run on the development machine and are left to CI.

Known limits

  • The 50-variable refusal carries the code DEPLOY.SERVICE_VARIABLE_LIMIT, but the CLI reports any failed plan as DEPLOY.ENGINE_FAILED, so the user sees that headline with the real message beneath it. Fixing it is a change to the framework's CLI and is recorded as later work.
  • Composer cannot count variables a user set on a service in Console; they count toward the 50.

Alternatives considered

  • Send a service's complete set and have the platform delete the rest. Rejected: it deletes values a user set in Console.
  • Leave unused keys stored on the service. Rejected in design review: a stale secret would reach every later deployment, and a service that reached 50 could not be deployed or cleaned up.
  • Have the provider read the service's current keys and delete those under a prefix. Rejected: a second prop on the upstream resource, one more call per deployment, and a dependency on the platform's read listing a service's keys correctly.
  • One variable holding all of a service's keys as JSON. Rejected: a value is limited to 8,192 characters, and it changes how every service boots.
  • Keep the pull request a draft with no check. Rejected in code review: nothing would stop a release if it merged early.

This is the third of three pull requests that make Composer work with variable groups. The decision is recorded in ADR-0069. The design is in .drive/projects/variable-groups-compat/design.md.

Agent: democritus-22

🤖 Generated with Claude Code

wmadden-electric and others added 15 commits November 20, 2026 23:41
…s with its deployment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…n its service

The platform is adding a variables field to the create-deployment request: a string sets a key, null deletes it, and a key not sent keeps its stored value. Neither @distilled.cloud/prisma nor @alchemy.run/prisma knows it yet, so both are patched with pnpm patch at the pinned versions.

Prisma.Deployment gains a variables prop. It is sent only on create and recorded as a salted hash. A changed, unresolved or newly declared map replaces the deployment, because an update creates no deployment and the values would never reach the platform.

The same change, as source diffs against the upstream repositories, is under .drive/projects/variable-groups-compat/slices/upstream for the upstream pull requests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…ment, and a deploy writes no variable rows

Every key a service reads at boot is sent in the variables of its create-deployment request, as its value or as null, so a value an earlier deploy stored, such as a streams key whose last consumer is gone, is deleted. A per-service SentKeys record in deploy state holds every key name ever sent, and each one this deploy no longer sends goes as null. A service with more than 50 variables is refused before anything changes.

The deployment still waits for its producers, and its triggers keep only the platform variable timestamps. The key-clash preflight check is deleted: with no rows written, two rows cannot collide.

On the first deploy with this version each deployment is replaced once and Alchemy deletes the old rows. A row whose platform record no longer matches deploy state is dropped from state with a warning naming its key, so it cannot fail every later deploy.

reserve, which only prisma dev runs, keeps writing local rows for a dev server, never for a null, and fails outside the local application.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
… the deployment and replace nothing when unchanged

The real-engine harness covers an unchanged redeploy, a changed value, a removed dependency's keys sent as null on that deploy and later ones, the streams key, producer ordering, the 50-variable refusal, no write to /v1/environment-variables, and the upgrade from rows: one replacement per service even with unchanged triggers, rows deleted afterwards, an interrupted upgrade, a row already gone, and a mismatched row dropped with a warning. Provider-level tests drive the patched Prisma.Deployment. Tests that asserted on rows now assert on variables, and the key-clash tests are deleted with the check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
… boots with its deployment's variables

The local Prisma.Deployment provider builds a service's environment from its variables prop, leaving out a key sent as null, in place of env.json. The emulator's port is still laid over the service's own port key, and the platform variables still come from the app configuration and secrets. A changed variables prop is a changed prop, so a changed value, an added key and a key turned to null restart exactly the service that reads it.

A dev server keeps reading the rows reserve writes to env.json, scoped to its own address. The converge test runs the real engine, the local providers and a real Compute emulator, and covers restarts, the dev server's rows, and a service switching between the two modes in both directions.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…eflight pull request words them

The same text as on feat/preflight-reads-resolved-variables, so the two branches merge cleanly; the next commit corrects what this branch makes false.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…e variables

ADR-0069 records the decision: what is sent, why every key boot reads is a value or null, the SentKeys record and the ways of deleting keys that were rejected, the limit of 50, what deploy state holds, the dev server's rows, the first deploy of an existing app, and that going back to an older Composer is not supported.

ADR-0048, ADR-0065, ADR-0041, ADR-0057, ADR-0056, ADR-0032 and ADR-0039 are rewritten in place where they described rows, the row ordering edge, trigger members that repeated values, or the key-clash check. The lowering, build reporting, platform model, config params, local dev and core model docs, the glossary, and the deploying and running-locally guides follow. The design and the deferred list record what was built differently and what is left.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…rkspace patches

A pnpm patch applies only to this workspace's install. A user who installs @prisma/composer-prisma-cloud gets the unpatched @alchemy.run/prisma, which drops a deployment's variables without an error, so every service would boot without its COMPOSER_ keys while the deploy reports success.

check:patched-dependencies fails when a published package, or a workspace package it installs, depends at run time on a package in pnpm.patchedDependencies. It runs in the CI Lint job and before publishing. It fails on this branch until both upstream packages release the variables field and the patches are removed.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
… the deployment

The variables doc comment now says the values are stored on the App and outlive the deployment, and that every Deployment of one App writes the same store. The helper that reads a recorded hash, used for triggers and variables alike, is persistedHash. The proposed upstream diff is regenerated from the same source.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
… 50-variable refusal as DEPLOY.SERVICE_VARIABLE_LIMIT

SentKeys is SentVariableNames (type PrismaCloud.SentVariableNames, role sent-variable-names), so it does not read as credentials; no release has written it to deploy state. ComputeSerialized carries bootVariables, every key boot reads, and requestVariables, what the create-deployment request carries with a null for each name Composer no longer sends. The provider wrapper is providerForDeletingOldVariableRows.

The refusal of a service with more than 50 values is a structured error with code DEPLOY.SERVICE_VARIABLE_LIMIT, naming the service, the count and the limit, and saying that no resource has been created, changed or deleted. Stale comments are cut, three single-file exports are private, the ordering test checks the consumer's deployment, and a test shows that the deploy after a name stops being sent replaces nothing.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…lease conditions, stated as they are

ADR-0069 and the deploying guide say a user can change a service's variables in Console and that Composer sets its own keys again on every deploy that creates a deployment; that a key losing its value in the upgrade deploy is still read from its old row; and that the 50-variable refusal comes before any resource is created, changed or deleted but after the topology is replaced. ADR-0044 registers DEPLOY.SERVICE_VARIABLE_LIMIT. ADR-0064 has a service scope. The data model gives the platform's real order of steps and source. The glossary and core model lose lines this branch made wrong. The plan records that slice 3 merges after slice 2 and what must happen before it is ready. Touched prose is one paragraph per line.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…ngine failure

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…ckages wait for it too

The check now reads every declared patch (package.json, pnpm-workspace.yaml and the lockfile) and walks each published package's run-time install through pnpm-lock.yaml, so a patched package that only a third-party dependency pulls in is found. A workspace package that is only a devDependency is bundled, and its imports are inlined unless the published package lists them, so the walk skips it.

The pull-request preview workflow runs the check before publishing to pkg.pr.new, because a preview install gets no pnpm patch either.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
…isma dev

prisma dev refuses the same service, as a deploy would. Its why no longer claims the branch's topology was replaced and a build recorded, which only a deploy does.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
… deployed

After one deploy with this version, deploy state holds a PrismaCloud.SentVariableNames resource per service, and Alchemy stops while planning when no provider is registered for a type in state, so an older Composer fails before changing anything. ADR-0069, the guide, the index and the design say so, and the Console workaround is gone. The guide says how the 50-variable refusal reaches the user today, ADR-0044 says prisma dev raises it too, and the plan states the merge order once and adds the real-platform checks before the pull request is ready.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
@prisma-gizmo

prisma-gizmo Bot commented Nov 20, 2026

Copy link
Copy Markdown

⏸️ Gizmo skipped the automatic review of a1d1835 because this pull request is a draft. It will review automatically once the PR is marked ready for review.

  • Check this box to request a one-time review of the current commit.

@coderabbitai

coderabbitai Bot commented Nov 20, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant