feat: Add secret resolution system design for managed secrets - #750
induwara-yaala wants to merge 36 commits into
Conversation
…vironment fallback
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The design has unresolved scope, API, security, concurrency, error-contract, and deployment requirements.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
Open (8)
Key validation permits unsafe environment variable access · New Public types reject documented None default behavior · New Cache locking does not ensure provider thread safety · New cache_ttl lacks range validation and zero/negative semantics · New Terraform opt-in variable and resource gating are underspecified · New Example scope omits required AWS containerized deployment · New Design narrows approved requirements without documented scope change · New Stage 1 design includes premature implementation details · New
What changed in this PR
This PR adds a Stage 1 design for managed secret resolution with pluggable providers, SSM, environment fallback, caching, and AWS deployment wiring.
Changes:
- Defines manager/provider APIs, configuration, injection, and error handling.
- Specifies caching, IAM/Terraform integration, examples, and testing.
- Documents scope, non-goals, and open questions.
| File | Description |
|---|---|
docs/specs/749-secret-resolution/design.md |
Stage 1 managed-secret resolution system design |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…he TTL constraints, and concurrency safety - Enforced key grammar (`^[a-z][a-z0-9_]*$`) to validate at the API boundary. - Defined behavior for `cache_ttl`: `> 0` caches, `0` disables, `< 0` errors. - Clarified synchronization with a resolution-wide lock for single-flight cold key fetch. - Enhanced documentation on Terraform opt-in (`ssm_enabled`) with least-privilege policies for serverless and containerized deployments. - Updated examples to demonstrate SSM integration for both deployment modes.
- Introduced `agentkernel/secret/` package with layered secret resolution: cache, provider, and environment-variable fallback. - Documented core classes, factory patterns, and provider implementations (`noop`, `ssm`). - Enhanced error handling with distinct errors for backend failures and missing secrets. - Updated `StarburstManager` to consolidate environment variable reads at initialization. - Elaborated on consumer compatibility, concurrency, and deployment configuration.
…ownership - Updated secret key grammar to uppercase format (`^[A-Z][A-Z0-9_]*$`) and validated at API boundary. - Clarified provider-specific addressing, removing path composition from manager. - Adjusted `inject` behavior to write keys verbatim to `os.environ`. - Documented updated `awssm` provider with explicit prefix handling and lodash-case transformation. - Enhanced error handling clarity for empty prefixes and credential requirements.
…back - Detailed iteration-based plan for introducing layered secret resolution. - Covers configuration, capability core, provider support (`noop`, `in_memory`, `env`, `awssm`), testing, and integration. - Documents Terraform changes and SSM integration for serverless and containerized deployments. - Updates examples, CI matrix, and cross-component tests for functional validation. - Includes steps to synchronize dev skills, bundled user skills, and docs for secret management.
…nvironment fallback
- Updated implementation plan and spec to prioritize environment variables over cache and provider. - Adjusted terminology from `awssm` to `aws_ssm` for consistency. - Enhanced examples, CI, and test coverage to reflect clear environment-first behavior. - Improved concurrency handling with distinct environment, cache, and provider locks.
…nd caching - Introduced `_SecretConfig` and `_SecretProviderConfig` classes for secret resolution. - Added configuration validation, default provider (`env`), and cache TTL support. - Extended `AKConfig` to include `secret` configuration. - Added tests to validate default behavior, environment overrides, and validation constraints.
- Implemented `SecretManager` for layered resolution: environment, cache, provider. - Added `SecretProviderFactory`, defaulting to `EnvSecretProvider` backend. - Introduced `SecretCache` with TTL configuration; supports eviction and invalidation. - Added unit tests to cover concurrency, cache behavior, environment overrides, and provider specifics. - Established error handling distinctions for provider failures (`SecretError`) and missing secrets (`SecretNotFoundError`).
- Introduced `SecretProviderContract` in a new `testing` module for reusable contract validation. - Added tests for `EnvSecretProvider` and `_DictSecretProvider` to ensure compliance. - Verified that `testing` module and pytest dependencies are excluded from main imports (`agentkernel.secret`).
…ctory/tests - Introduced `AWSSMSecretProvider` supporting AWS SSM Parameter Store as a managed secret store. - Updated `SecretProviderFactory` to support `aws_ssm` type with validation for prefix configuration. - Ensured lazy and singleton initialization of SSM clients for efficient use. - Added comprehensive tests for `AWSSMSecretProvider` contract compliance, error handling, and configuration validation. - Included factory-level tests for building `AWSSMSecretProvider` and validating dependency requirements.
… modules - Added `ssm_enabled` variable to enable read-only access to SSM Parameter Store secrets with scoped IAM permissions. - Updated Lambda functions to inject `AK_SECRET__PREFIX` into environment variables when `ssm_enabled` is true. - Applied changes across handler modules (request, response, agent, WebSocket connection) for consistent secret resolution. - Enhanced documentation and state management to include `ssm_enabled` configuration and behavior.
…ules - Added `ssm_enabled` variable to grant scoped IAM permissions for accessing SSM Parameter Store. - Updated ECS task roles for REST service and agent runner to support `ssm_enabled`. - Injected `AK_SECRET__PREFIX` into environments when `ssm_enabled` is enabled. - Enhanced Terraform with IAM policy for parameter access and updated documentation to reflect changes.
…ocumentation with detailed instructions for secret setup, resolution order, and key rotation.
…ate documentation for SSM Parameter Store integration
- Introduced `config.yaml` to define secret provider and cache TTL configuration. - Added `build.sh` for environment caching and syncing dependencies. - Included `uv.lock` with detailed dependency list for reproducibility.
…able and wildcarding ARN - Updated all resource ARNs to wildcard the account ID (`arn:aws:ssm:<region>:*:parameter/...`) for SSM access. - Removed unnecessary `account_id` variables from all modules, streamlining configuration. - Updated documentation to reflect the changes in SSM permissions and module setup.
… package version bumps
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Critical IAM scope, deployment secret injection, lock reproducibility, and validation issues remain unresolved.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 9
Open (14)
Scope SSM IAM ARN to the current AWS account · New Scope SSM IAM ARN to the current AWS account · New Scope SSM IAM ARN to the current AWS account · New Scope SSM IAM ARN to the current AWS account · New Scope SSM IAM ARN to the current AWS account · New Scope SSM IAM ARN to the current AWS account · New Restore API key injection or provision SSM parameter · New Restore API key injection or provision SSM parameter · New Regenerate lockfile for clean-checkout reproducibility · New Use full-string matching for secret key validation · New Update stale Terraform module version reference · New Complete documentation and skills synchronization · New Confirm design approval before advancing staged spec · New Update stale release-sequencing version reference · New
Resolved since last review (8)
Key validation permits unsafe environment variable access Design narrows approved requirements without documented scope change Example scope omits required AWS containerized deployment Terraform opt-in variable and resource gating are underspecified cache_ttl lacks range validation and zero/negative semantics Cache locking does not ensure provider thread safety Public types reject documented None default behavior Stage 1 design includes premature implementation details
amithad
left a comment
There was a problem hiding this comment.
Overall: solid, spec-driven implementation. The secret/ package, the _SecretConfig block, the Terraform wiring and the tests match the approved design and the house patterns (pluggable by default, justified config fields, classes not scripts, secret/ imports core only). The main gap is that the spec set no longer describes what shipped for the examples and CI. CI: all 64 checks pass.
Spec verdict (reviewed before the code)
design.md,spec.mdandplan.mdall reviewed. Everypath:line, default and count cited against the base branch verified (the 34main.tffiles, everyconfig.py, Terraform module and CI line reference), with one exception: the "70 models" count is 68 (nit inline).- Cross-document consistency holds:
spec.mdcovers everydesign.mdrequirement andplan.mdcovers everyspec.mdcomponent. The one break is the examples/CI decision, which changed during implementation and is now recorded in the code, READMEs and skills but in none of the three documents (inline onspec.md:811). - Design approval staging and the scope narrowing vs #749 were already raised by Copilot and are not repeated here.
Spec conformance (implementation)
- Implemented as specified:
SecretProviderABC, concreteSecretManagerwithcurrent()/reset(),SecretCache(monotonic TTL, lock-free reads, guarded eviction),SecretProviderFactorywithrequire_extra+resolve_dotted,env/aws_ssmproviders,SecretProviderContract;_SecretConfigwith exactly the three justified fields and noenabled/managertype; no module-level functions and nothing outsidesecret/imports it;ssm_enabledon both roots,count-gated, account-scopedssm:GetParameterper tier,AK_SECRET__PREFIXinjected, provider type never injected; docs and skills sync done except the new dev skill. - Deviated (inline): the two AWS examples drop the
OPENAI_API_KEYinjection instead of keeping it optional; CI now seeds SSM where the spec says "no CI change"; theak-dev-new-secret-providerskill required byplan.mdIteration 9 is absent. - Beyond the spec:
examples/cli/openai_secretand its e2e matrix entry (useful and green in CI). Worth one line inspec.mdExamples andplan.mdIteration 7 so the plan matches the PR.
Findings: 0 blockers, 4 suggestions, 2 nits, 1 question (all inline).
Not anchorable to a diff line
[suggestion]Tests:spec.mdTesting andplan.mdIteration 8 step 1 promise "the environment wins for every provider" and "an empty variable reachesaws_ssm" driven through the manager with the fake SSM client recording zero or oneget_parametercall. Only the dict-provider variants exist (test_environment_wins_over_cache_and_provider,test_empty_environment_variable_is_a_miss). Two small tests intest_secret_manager.pyusing the_FakeSSMClientfixture would close it.[suggestion]Docs-site React pages: README,intro.mdand the docs pages now list the capability, butdocs/src/pages/index.tsx(ak-add-capabilitiespills) andfeatures.tsx(Core Capabilities cards) do not. Scheduling and threads have no card either, so this is optional; aSecretspill on theak-add-capabilitiesentry is the cheapest consistency fix.
Positives
_KEY_PATTERN.fullmatchand the${var.account_id}-scoped ARNs address the earlier Copilot findings.- The
_DictSecretProvidertest double is itself held toSecretProviderContract; the cache eviction race has a deterministic test; no secret value can reach a log record or exception message; the CI seed passes the value via--cli-input-json file://, never argv.
Skipped as duplicates of existing feedback: design approval staging (spec.md:17), scope narrowing vs #749 (design.md:167), Stage-1 implementation detail (design.md:88).
…ssm example to remove direct OpenAI API key injection - Introduced `ak-dev-new-secret-provider` skill with step-by-step guidance for extending secret resolution. - Updated AWS examples to exclusively use AWS SSM for OpenAI API key resolution, eliminating Terraform state secrets. - Added CI seeding for SSM parameters in affected examples to validate the SSM path during tests. - Adjusted CI roles for `ssm:PutParameter` permission and documented changes in READMEs and dev skills
…es over managed store
# Conflicts: # .agents/skills/ak-dev-architecture/SKILL.md # docs/docs/core-concepts/configuration.md
…nd documentation for consistency
# Conflicts: # ak-py/src/agentkernel/skills/ak-add-capabilities/evals/evals.json
…nd documentation for consistency
…solve Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…vider` and related documentation
…idate logic into `current()`
…ments across modules
…oved OpenAI client handling
…SMSecretProvider`



Description
Add secret resolution system design for managed secrets
Type of Change
Changes Made
Add secret resolution system design for managed secrets
Testing
Checklist