Skip to content

Add lambda-cloudwatch-otlp-metrics-sam pattern - #3298

Open
manishh-13 wants to merge 1 commit into
aws-samples:mainfrom
manishh-13:manishh-13-feature-cloudwatch-otlp-metrics
Open

Add lambda-cloudwatch-otlp-metrics-sam pattern#3298
manishh-13 wants to merge 1 commit into
aws-samples:mainfrom
manishh-13:manishh-13-feature-cloudwatch-otlp-metrics

Conversation

@manishh-13

@manishh-13 manishh-13 commented Sep 2, 2026

Copy link
Copy Markdown

Issue #, if available:

Submission issue: #3299

Description:

Adds a new pattern: lambda-cloudwatch-otlp-metrics-sam.

Records custom application metrics in an AWS Lambda function using the OpenTelemetry metrics API and sends them to Amazon CloudWatch over OTLP. The AWS Distro for OpenTelemetry Lambda layer runs a collector next to the function that signs each request with SigV4 and forwards it to the CloudWatch OTLP endpoint. The function makes no PutMetricData calls, writes no embedded metric format logs, and performs no request signing of its own. The only IAM permission it needs is cloudwatch:PutMetricData.

Why OTLP rather than PutMetricData or embedded metric format:

  • No per series charge, so high cardinality metrics are billed on volume instead of on the number of unique dimension combinations.
  • Up to 150 labels per metric, against the 30 dimensions allowed by PutMetricData.
  • No synchronous AWS API call inside the invocation path.

The pattern also documents the details that are easy to get wrong, all of which I hit while testing:

  • The collector build in the ADOT Lambda layer is compiled without processors, so naming one such as batch stops the collector from starting and nothing is exported.
  • The exporter must use metrics_endpoint, not endpoint. The exporter appends the signal path to endpoint, so setting endpoint to the full metrics URL produces /v1/metrics/v1/metrics and every export fails with HTTP 404.
  • The function flushes the meter provider before returning, because Lambda freezes the execution environment as soon as the handler returns.
  • The CloudWatch OTLP endpoint is off by default and must be enabled once per account and Region, which the README covers.
  • These metrics are queried with PromQL and do not appear in the classic CloudWatch metrics list.

Language: Python 3.13. Framework: AWS SAM. Level: 300.

Tested end to end in a live account: deployed with sam build and sam deploy, invoked the function, confirmed the collector exported with no errors, and queried the CloudWatch PromQL endpoint to verify the data points arrived with the expected value and labels. The stack was then deleted.

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT-0 license.

Record custom application metrics in AWS Lambda with the OpenTelemetry metrics
API and send them to Amazon CloudWatch over OTLP. The ADOT Lambda layer runs a
collector that signs each request with SigV4, so the function makes no
PutMetricData calls and writes no embedded metric format logs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants