Skip to content

fix: redact credential-like fields in meta before logging - #108

Open
anrenlx2025 wants to merge 1 commit into
singchia:mainfrom
anrenlx2025:fix/redact-meta-logging
Open

anrenlx2025 wants to merge 1 commit into
singchia:mainfrom
anrenlx2025:fix/redact-meta-logging

Conversation

@anrenlx2025

Copy link
Copy Markdown

Problem

The frontier server logs the full connection meta (edgebound/servicebound/exchange, 24 call sites) on edge/service online/offline, heartbeat, stream and forward paths. Clients commonly carry credentials (e.g. access_key/secret_key) in meta, so every connect/disconnect cycle writes plaintext credentials into system logs. Several of these sites are klog.Errorf (unconditional), the rest are klog.V(1..3). In addition, the geminio SDK's default logger prints raw meta to stdout on close-handshake error paths, bypassing klog entirely.

Fix

Add misc.Redact(meta string) string, wrap all 24 meta log sites, and close the SDK logging bypass:

  • Parses the JSON meta and replaces the value of any key whose lowercased name contains a credential-like fragment (key / secret / token / password / passwd / pass / pwd / credential / auth / signature / bearer / session / cookie / cert / private) with ***, recursively (nested objects, objects inside arrays). Non-string values under sensitive keys are also masked. Substring matching intentionally errs on the side of over-masking: innocuous keys such as keyword or author are masked too, which is preferred over leaking.
  • Fail-closed: non-JSON input returns <redacted: unparsable meta>; input larger than 8KB returns <redacted: meta too long>; JSON whose top-level value is not an object/array (e.g. a bare string, which often carries an already-serialized object) returns <redacted: non-object meta>. Raw meta is never echoed.
  • Wire the geminio SDK end logger through klog (opt.SetLog(log.NewKLog())) for edgebound and servicebound ends, mirroring the existing wiring in frontlas/frontierbound: with the default logger, the SDK prints raw connection meta to stdout on close-handshake error paths, which would bypass the redaction above.
  • Table-driven tests (24 cases) cover masking variants, case-insensitivity, nesting, non-string sensitive values, non-JSON, double-encoded JSON, top-level scalars, trailing garbage, over-redaction acceptance, and the exact 8KB boundary.

Known boundaries (intentionally out of scope)

  • Meta stored in the repo (DB), returned by control-plane APIs, or forwarded to frontlas is functional data flow, not logging; unchanged.
  • meta.Service in the service forward path is a plain routing name parsed from validated JSON, not a credential carrier; logged as is.
  • Stream/connection-level meta that is not valid JSON is replaced by a placeholder as a whole (fail-closed by design).
  • Redact runs even when klog verbosity filters the line out (argument eager evaluation); measured cost (~10-90us for typical <1KB to 8KB meta) is negligible for typical fleet sizes, kept simple rather than wrapping 24 sites in klog.V(n).Enabled() guards.

Test plan

  • go test ./pkg/frontier/misc/ (table-driven tests, 24 cases)
  • go build ./... on both windows and linux, go vet on touched packages
  • gofmt blob-level clean on all 9 changed files
  • Pre-existing failure note: pkg/frontier/edgebound TestEdgeManagerStream panics at newEdgeManager on both v1.2.5 baseline and this branch (unrelated to this change, reproduced via stash)

The frontier server logs the full connection meta (edgebound/servicebound/
exchange, 24 call sites) on edge/service online/offline, heartbeat, stream
and forward paths. Clients commonly carry credentials (e.g. access_key/
secret_key) in meta, so every connect/disconnect cycle writes plaintext
credentials into system logs.

Add misc.Redact(meta string) string and wrap the meta log sites:

- Parse the JSON meta and replace the value of any key whose lowercased
  name contains a credential-like fragment (key/secret/token/password/
  passwd/pass/pwd/credential/auth/signature/bearer/session/cookie/cert/
  private) with "***", recursively (nested objects, objects inside
  arrays). Non-string values under sensitive keys are also masked.
  Substring matching intentionally errs on the side of over-masking:
  innocuous keys such as "keyword" or "author" are masked too, which is
  preferred over leaking.
- Fail-closed: non-JSON input, oversized input (>8KB), and JSON whose
  top-level value is not an object/array all return a placeholder without
  echoing the raw value. A top-level string often carries an
  already-serialized object, so it is masked as a whole.
- Wire the geminio SDK end logger through klog (SetLog) for edgebound and
  servicebound ends, mirroring the existing frontlas wiring: the SDK
  default logger prints raw connection meta to stdout on close-handshake
  error paths, which would bypass the redaction above.
- Table-driven tests cover masking variants, case-insensitivity, nesting,
  non-string sensitive values, non-JSON, double-encoded JSON, top-level
  scalars, trailing garbage, over-redaction acceptance, and the exact
  8KB boundary.

Known boundaries (intentionally out of scope): meta stored in the repo,
returned by control-plane APIs, or forwarded to frontlas is functional
data flow, not logging; unchanged. meta.Service in the service forward
path is a plain routing name parsed from validated JSON, not a
credential carrier, and is logged as is.
@vercel

vercel Bot commented Sep 13, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the singchia's projects Team on Vercel.

A member of the Team first needs to authorize it.

@anrenlx2025

Copy link
Copy Markdown
Author

Thanks for reviewing when you get a chance. Since this is my first contribution to this repo, the CI workflow on the fork PR needs maintainer approval to run (action_required) — could you approve the workflow run so the build check can execute? For reference, I've rehearsed the same go build ./... workflow on my fork and it passes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant