Skip to content

awsutil: LoadSecretBundle(ctx) — opt-in Lambda env-cap reclamation (D5 of sc-secrets-bundle-mode) - #20

Merged
smecsia merged 1 commit into
mainfrom
sdk/load-secret-bundle
Oct 17, 2026
Merged

smecsia merged 1 commit into
mainfrom
sdk/load-secret-bundle

Conversation

@smecsia

@smecsia smecsia commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Fifth slice (D5) of the sc-secrets-bundle-mode initiative — see forge/docs/design/2026-10-17/sc-secrets-bundle-mode/README.md for the full design.

Adds LoadSecretBundle(ctx) to pkg/awsutil/secrets.go: reads a FORGE_SECRETS_BUNDLE_ARN env (the SecretsBundleEnv constant), fetches the JSON payload via the existing secretcache.Cache, parses it as {envName:string → value:string}, and os.Setenv's each key.

SC's Pulumi renderer (D3/D4 in the api repo) is the sibling change that writes the FORGE_SECRETS_BUNDLE_ARN env + the ONE Secrets Manager JSON secret per service. Together, N per-key ARN envs become 1 — reclaims ~4-5 KB on the forge-conductor Lambda, which has been at the 4KB env cap since 2026-10-16.

Contract preserved for the rest of the fleet

  • Unset FORGE_SECRETS_BUNDLE_ARN = NO-OP (nil error). This SDK version can be pinned before any service opts in on the SC side. Legacy per-key GetEnvOrSecret works unchanged.
  • Set-but-not-ARN = loud error. is set but is not a Secrets Manager ARN (got prefix "..."). Operator typos surface instead of loading garbage.
  • Operator-pinned env values win. os.Setenv skips keys already set in the environment — anything the operator pinned via env: in client.yaml overrides the bundle.
  • Non-string JSON values rejected. Explicit error rather than silent Go-default stringification.
  • Shared cache. Rides the same lazy secretcache.Cache GetEnvOrSecret uses. No second AWS session. On Yandex (no bundle env, no AWS creds) the cache stays unbuilt.

Also adds LoadSecretBundleFromEnv(ctx, envName) for services that need a different bundle env name during staged migrations.

Call-site contract

Services opt in by adding ONE line to the top of main():

if err := awsutil.LoadSecretBundle(context.Background()); err != nil {
    log.Fatalf("secret bundle load: %v", err)
}

Everything downstream continues to call GetEnvOrSecret / os.Getenv unchanged.

Test plan

  • TestLoadSecretBundleUnsetIsNoop — unset bundle env is a no-op (nil error)
  • TestLoadSecretBundleRejectsNonArn — garbage value surfaces as "not a Secrets Manager ARN"
  • TestLoadSecretBundleFromEnvRespectsExplicitEnvName — explicit env-name variant works
  • TestFirstN — error-message truncation helper
  • Existing TestGetEnvOrSecretReturnsLiteralWithoutTouchingAws + TestGetEnvOrSecretReturnsEmptyForUnsetVariable still pass
  • go vet + go build + gofumpt -w + gci write applied
  • Live-smoked once the api-side D2+D3 lands and forge-conductor D6 pilot runs

References

  • Design doc: forge/docs/design/2026-10-17/sc-secrets-bundle-mode/README.md
  • Roadmap item: forge/docs/roadmap/items/sc-secrets-bundle-mode.md (slice D5)
  • Blocked siblings waiting for env-cap reclamation: forge-conductor conductor-lambda-env-4kb-cap, Resend webhook pivot PR #601 workaround

New helper that reads a FORGE_SECRETS_BUNDLE_ARN env (constant
SecretsBundleEnv), fetches the JSON payload via the existing
secretcache.Cache, parses it as {envName:string → value:string},
and os.Setenv's each key before the service's main() reads a
secret env.

The design (SC + SDK coordinated change — see forge/docs/design/
2026-10-17/sc-secrets-bundle-mode/README.md) uses this to replace
N per-key Secrets Manager ARN env vars with ONE bundle env on
Lambda stacks that hit the 4KB env cap. SC's Pulumi renderer is
the sibling change in the api repo.

Contract preserved for the rest of the fleet:
  - Unset FORGE_SECRETS_BUNDLE_ARN = NO-OP (nil error). Services
    can pin this SDK version before any opt in on the SC side.
  - Set-but-not-ARN = loud error (`not a Secrets Manager ARN`).
    Operator typos surface instead of loading garbage.
  - Operator-pinned env values win over bundle values (os.Setenv
    skips keys already set). Literal env always wins.
  - Non-string JSON values are rejected explicitly rather than
    silently flattened via Go's default string format.
  - Shares the SAME lazy secretcache.Cache GetEnvOrSecret uses;
    no second AWS session on Yandex where no bundle is wired.

Also adds LoadSecretBundleFromEnv for services that need a
different bundle env name (staged migrations).

Four hermetic tests pin the unset-noop, non-ARN rejection,
explicit-env-name variant, and the error-message truncation
helper.
@smecsia
smecsia merged commit 44dbeb7 into main Oct 17, 2026
4 of 5 checks passed
@github-actions

Copy link
Copy Markdown

Semgrep Scan Results

Repository: go-aws-lambda-sdk | Commit: dd5d959

Check Status Details
🚨 Semgrep ERROR 1 error(s), 2 warning(s), 3 total

Scanned at 2026-10-17 13:57 UTC

@github-actions

Copy link
Copy Markdown

Security Scan Results

Repository: go-aws-lambda-sdk | Commit: dd5d959

Check Status Details
✅ Secret Scan Pass No secrets detected
❌ Dependencies Failed Check workflow logs

Scanned at 2026-10-17 14:00 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant