Repository navigation
Conversation
Deletes pkg/clouds/gcloud/ + pkg/clouds/pulumi/gcp/ (13,325 LOC) from
api core. sc-plugin-gcp (template-complete at main 8bf6d5d, 10 tokens
across 7 resources + 3 templates) now owns every GCP type-token
authoritatively, same shape sc-plugin-{aws,yandex,cloudflare,cloudru,
mongodb} already follow.
Changes:
- `pkg/clouds/gcloud/` — DELETED (402+3197+244+19+862+327 LOC across
the config tier + provisioning tier). sc-plugin-gcp carries these
verbatim lifted in phases 2 through 4g.
- `pkg/clouds/pulumi/gcp/` — DELETED (10 .go files + 3 test files).
- `pkg/clouds/pulumi/init.go`: dropped `_ "pkg/clouds/pulumi/gcp"`
blank-import.
- `pkg/provisioner/init.go`: dropped `_ "pkg/clouds/gcloud"` blank
import with a comment pointing at the plugin.
- `pkg/clouds/pulumi/pulumi.go`: `StateStorageTypeGcpBucket` now
literal `"gcp-bucket"` instead of pulling `gcloud.
StateStorageTypeGcpBucket` — same shape S3Bucket already takes one
line below.
- `pkg/clouds/pulumi/kubernetes/naming.go`: GKE Autopilot special-case
in `envCaddyObjectsName` / `deploysEmbeddedCaddy` lifted off the
gcloud typed struct onto a local `gkeAutopilotProbe` with the two
fields actually read (Caddy, Adopt). Token hardcoded as a local
const. Decouples the shared kubernetes package from sc-plugin-gcp's
/api without introducing an extension-point refactor.
- `pkg/clouds/pulumi/kubernetes/caddy_naming_test.go`: mirror test
fixture struct + const.
- `pkg/clouds/pulumi/kubernetes/namespace_threading_test.go`: dropped
the gcp/compute_proc.go lint entry — the invariant it guards
belongs on sc-plugin-gcp's test suite now.
- `pkg/api/plugin_bootstrap.go`: added sc-plugin-gcp canonical row
(every token + github-release source) alongside the kind-aware
`RegisterPluginOwnedType` block.
- `cmd/schema-gen/main.go`: dropped the gcloud blank-import, added
`_ "sc-plugin-gcp/api"` alongside the aws/mongodb/yandex pattern.
- `pkg/api/tests/refapp*.go`, `pkg/clouds/pulumi/e2e_*_test.go`,
`pkg/clouds/pulumi/testutil/secrets_test_util.go`,
`pkg/provisioner/scoped_deploy_test.go`,
`pkg/provisioner/placeholders/tests/placeholders_test.go`: swapped
typed gcloud imports to `gcpApi "sc-plugin-gcp/api"` since the
types were lifted name-for-name.
- `go.mod`: `sc-plugin-gcp/api` pinned to v2026.11.07-8bf6d5d (the
fix bundle carrying deletionProtection + md5 URN suffix).
Open test slice: shape-of-config tests under
`pkg/api/tests`, `pkg/cmd/cmd_secrets`, `pkg/provisioner`,
`pkg/e2e/scopedstore` assert against typed gcloud configs; with the
in-tree path gone, server.yaml parsing produces
`*api.PluginFieldConfig` via the opaque plugin path. Tests need
updating to assert the new shape. Not a production-code regression —
separate follow-on slice.
Blast radius: additive for operators on sc-plugin-gcp, breaking for
anyone who imports `pkg/clouds/gcloud` or
`pkg/clouds/pulumi/gcp` directly. Those two package paths were
documented API surfaces of api core, so this is the first
deletion-wave release — bump CalVer appropriately.
Roadmap: closes `sc-plugin-gcp-extraction` phase 5 (the api PR).
Phase 6 (fleet forge pin bump) + phase 7 (migration-safety smoke)
follow once this merges + an api CalVer release cuts.
Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
Semgrep Scan ResultsRepository:
Scanned at 2026-11-08 18:22 UTC |
Security Scan ResultsRepository:
Scanned at 2026-11-08 18:23 UTC |
…-import
Follow-up to the phase 5 delete commit — un-draft gate:
Four test fixture surfaces needed explicit `plugins: gcp:` blocks (or a
blank-import of sc-plugin-gcp/api in-test) so the test binary can
resolve GCP type-tokens without a GitHub latest-release lookup. Without
this, CI runners that don't have GitHub API quota hit a 403 Forbidden
and the plugin auto-inject fallback fails.
Test fixtures updated:
- pkg/e2e/scopedstore/scoped_e2e_test.go: inline serverYAML gains
plugins block.
- pkg/provisioner/scoped_deploy_test.go: scopedParentServer const
gains plugins block.
- pkg/api/tests/testdata/stacks/{refapp,refapp-gke-autopilot,refapp-
static-gcp,refapp-ansible-k3s,common}/server.yaml: plugins block
inserted alongside the existing aws/cloudflare plugin declarations.
- pkg/api/{git,secrets}/testdata/repo/stacks/common/server.yaml: same.
- pkg/cmd/cmd_secrets/cmd_scope_hardening_test.go: blank-import
sc-plugin-gcp/api so its init() registers typed decoders for
gcp-service-account — restores the parse-time strictness
TestScopeCmd_LintChecksAuthEntriesWithEnvScopeKeysOnly relies on.
newScopeRepo also writes a minimal plugins-declaring server.yaml.
Also bumps go.mod to sc-plugin-gcp/api v0.0.0-20261108045141-58ce4b6cfa4a
(pseudo-version for the `RegisterProvisionerFieldConfig` fix that
closes the typed-state-storage / typed-secrets-provider gap for
TestReadServerDescriptor).
Verified: `go test ./... -count=1 -short` passes cleanly on the branch.
Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
smecsia
marked this pull request as ready for review
November 8, 2026 05:03
smecsia
requested review from
Cre-eD,
Laboratory and
universe-ops
as code owners
November 8, 2026 05:03
…ispatch Mirror the parent_info_json field that ComputeProcessRequest carries (:391) onto ProvisionRequest. A plugin template whose Provision body reaches into params.ParentStack.FullReference — gke-autopilot reading kubeconfig + registry URL, static-website reading zone HMAC creds — needs the parent-stack reference across the bus, and today the plugin dispatch route silently drops it. Live-caught on sc-plugin-gcp phase 4g tier-3 smoke 2026-11-07: GkeAutopilotStack:54 SIGSEGVs on params.ParentStack.FullReference because SC core never populates ProvisionParams.ParentStack when routing a template Provision through the plugin dispatcher. Parent-side provision was GREEN (21 real GCP resources), crash fired in the plugin process. Same class as the YC "green deploy, zero revisions" trap. Changes: - proto: new `bytes parent_info_json = 11` on ProvisionRequest with doc comment pointing at the roadmap item for the two-guardrails framing. - sidecar: makeResourceSidecar's ProvisionRequest literal populates the field from params.ParentStack via json.Marshal when non-nil — same shape as the ComputeProcess sidecar one block below. - runtime: new ParentInfoFromProvisionRequest helper sits next to the existing ParentInfoFromRequest (ComputeProcess). Decodes JSON, returns (nil, nil) on empty bytes, wraps the message with the "provision request" prefix so a decode error names the right RPC. - tests: round-trip, empty-means-nil, malformed-rejection. Additive proto field — older plugins ignore, older cores send nothing, newer plugins get the same silent-drop behaviour they tolerate today (a nil ParentStack). The second guardrail (nil-checks in every template body that reads params.ParentStack) lands separately on each plugin; it's cheap (an errors.Errorf) and the two must ship together. See forge/docs/roadmap/items/sc-plugin-provisionrequest-missing-parent-info.md for the full framing + the parallel template-parentstack-nilcheck-lint roadmap row. aws-templates sidestep today by not reading params.ParentStack at all (grep shows zero uses in sc-plugin-aws/plugin/internal). The gap is invisible there, which is why it was invisible for six months of plugin-era deploys — until a GCP template that needs parent outputs actually ran. Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
Cre-eD
previously approved these changes
Nov 8, 2026
Phase 5 of the GCP extraction deletes pkg/clouds/gcloud, so the 10 `pkg/clouds/gcloud/*.go` lines in the reference become dead links the moment this PR merges. Rewrite them to point at `github.com/simple-container-com/sc-plugin-gcp/api/<file>.go`, which is where every one of those structs lives now. Also for the Cloud Run section: - The schema file was renamed by schema-gen from `cloudruninput.json` to `cloudruntemplate.json` when the plugin added `CloudRunTemplate` (the parent-template config operators write in server.yaml) on top of the pre-existing `CloudRunInput` (the deployment-time payload). Update the link and the struct reference accordingly — operators want to know what to put in `templates.foo.config`, that's CloudRunTemplate. - The yaml example was pre-Cloud-Run-v2 and didn't include the `region` or `artifactRegistryResource` fields the Provision body now requires. Add both to the example and a short note on the three behaviours live-smoked this week: auto-enable of `run.googleapis.com`, `deletion_protection: false`, and the public `allUsers → roles/run.invoker` default. Archived designs under `docs/design/archive/` and the dated `docs/design/2026-03-25/...` + `docs/implementation/2026-03-25/...` trees still reference the old paths; those are frozen historical records of past slices, not live reference docs, so they stay as-is. Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
universe-ops
force-pushed
the
feat/plugin-gcp-extraction-phase5
branch
from
November 8, 2026 12:41
7f8268c to
7e9abcf
Compare
smecsia
enabled auto-merge (squash)
November 8, 2026 12:42
Two consecutive #568 CI runs timed out on `TestScopeRegression_*` + `Test_Provision/happy_path_gcp` with: load schema for plugin "gcp": resolve plugin binary: download ".../releases/download/v2026.11.07-8bf6d5d/sc-plugin-gcp-linux-amd64": context deadline exceeded (Client.Timeout or context cancellation while reading body) The pinned release's binary is ~230MB and the plugin fetcher's default timeout is tight on GHA runner bandwidth to GitHub releases. The first rerun (SHA 2f7524a9 wanted, 0742299d got) was transient — the second is a clean timeout, meaning the fetch IS reliably touching the server but the body stream is slower than the cutoff. Bumping the pin to the latest sc-plugin-gcp release `v2026.11.08-d14f4da` (cut 2026-11-08 15:55 UTC, carries the ApplyT fix for `scCoreRegistrar.NewRecord` + all prior phase-5 fixes) doesn't shrink the binary — but it does route through a more recent artifact that's likely to have an active CDN edge cache from today's release workflow, improving cold-fetch latency on CI. If this bump doesn't clear the timeout, the fix lives on the fetcher side (add a longer timeout for first-of-session downloads, or ship a cache-pre-population action in the test setup). Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
2 of 3 tasks
smecsia
added a commit
that referenced
this pull request
Nov 8, 2026
…arge binaries (#575) ## Summary Two coupled timeouts bit #568 in CI: the plugin fetcher's 5-min HTTP client cutoff and `go test`'s 10-min per-package budget. sc-plugin-gcp is ~230MB (pulumi-gcp + pulumi-kubernetes + google-api-go-client), and a cold fetch on GHA hosted runners brushes up against both. ## What - **`pkg/plugin/fetcher.go`**: raise `downloadFile`'s HTTP client timeout 5 min → 15 min. Named const `downloadFileHTTPTimeout` with the incident citation inline. - **`.github/workflows/{branch,branch-preview,push}.yaml`**: raise the `go test ./...` step's budget to `-timeout 30m` (matching the already-higher budget on `test-registry-integration`). ## Live-caught api PR #568 (phase 5 of the sc-plugin-gcp extraction), 2026-11-08. Two consecutive reruns of `TestScopeRegression_DamagedScopeFileFailsLintDeployAndReseal` and `Test_Provision/happy_path_gcp` timed out mid-download with `context deadline exceeded (Client.Timeout or context cancellation while reading body)`. Bumping the fixture version pin to the latest release (v2026.11.08-d14f4da) didn't help — the binary size is the same. ## Why not pre-warm cache or stub the fetcher - **Pre-warm** (add a `curl` step before `go test`): workable, but adds a second code path that goes stale when the fixture pin bumps. The timeout bump handles every version pin automatically. - **Stub the fetcher in tests**: cleanest long-term, but requires refactoring TestScopeRegression + Test_Provision to use a path: spec with a built-at-test-time stub binary — larger scope, blocks #568's merge. Follow-up item worth filing separately. ## Test plan - [x] \`go build ./...\` - [x] \`gofumpt\` + \`gci\` applied - [ ] \`go test -timeout 30m ./...\` on #568 completes cleanly Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
…imple-container-com/api into feat/plugin-gcp-extraction-phase5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Deletes
pkg/clouds/gcloud/+pkg/clouds/pulumi/gcp/(13,325 LOC) from api core.sc-plugin-gcp(template-complete at main8bf6d5d, 10 tokens across 7 resources + 3 templates) now owns every GCP type-token authoritatively — same shape sc-plugin-{aws,yandex,cloudflare,cloudru,mongodb} already follow.Draft-only
Shape-of-config tests under
pkg/api/tests,pkg/cmd/cmd_secrets,pkg/provisioner,pkg/e2e/scopedstoreassert against typedgcloud.*configs. With the in-tree path gone, server.yaml parsing produces*api.PluginFieldConfigvia the opaque plugin path. These tests need updating to assert the new shape — not a production-code regression, but needs a separate slice before un-draft.Production code:
go build ./...+go vet ./...clean.What
pkg/clouds/gcloud/+pkg/clouds/pulumi/gcp/entirely.pkg/clouds/pulumi/init.go+pkg/provisioner/init.go+cmd/schema-gen/main.go— gcloud/pulumi/gcp removed,sc-plugin-gcp/apiadded as the equivalent in schema-gen.pkg/clouds/pulumi/pulumi.go(StateStorageTypeGcpBucketbecomes a literal),pkg/clouds/pulumi/kubernetes/naming.go(GKE Autopilot probe lifted off typed struct onto a localgkeAutopilotProbewith the two fieldsenvCaddyObjectsNameactually reads).pkg/api/plugin_bootstrap.goadds the sc-plugin-gcp row + the kind-awareRegisterPluginOwnedTypeentries for all 10 tokens.gcpApi "sc-plugin-gcp/api"since sc-plugin-gcp lifted the types name-for-name.Breaking changes
pkg/clouds/gcloudorpkg/clouds/pulumi/gcpbreaks. Those were documented API surfaces of api core — this is the first deletion-wave release, bump CalVer appropriately.sc-plugin-gcp(viaplugins:in server.yaml) is now the ONLY path to GCP.plugin_bootstrap.go's canonical row auto-injects for operators who paste an unknown-but-canonical GCP token.Stack
Independent of the #562-#566 wave (
feat/vm-multi-host-rolesfamily) and of #567 (feat/provisionrequest-parent-info-json). Based on main.Roadmap
sc-plugin-gcp-extractionphase 5.Test plan
go build ./...+go vet ./...cleangh workflow run branch.yaml --ref feat/plugin-gcp-extraction-phase5