Intermittent missing data for embedded value

[davidkopp]

The eks_node_embedded_value metric is intermittently missing from Prometheus. It is available for 5 minutes, disappears for 10 minutes, then becomes available again, repeating this cycle.

Investigation

• The metric is reset every 5-minute:
  

  keit/source/keit.go

  Line 88 in 861d3f8

  embeddedValueGauge.Reset()

  
  

  keit/source/keit.go

  Line 162 in 861d3f8

  time.Sleep(300 * time.Second)

• The ServiceMonitor is configured with an interval of 15m:
  

  keit/helm/values.yaml

  Line 24 in 861d3f8

  interval: 15m

Proposed Solution

Reduce the interval in the ServiceMonitor configuration to a value shorter than 5 minutes, e.g. 1m

[davidkopp]

I tested setting the scrape interval to 5m and it seems to be sufficient:

[geurjas]

Can I close this one? think its solved by merging your pull request.

[davidkopp]

The merged pull request hasn't changed the scrape interval. It's still set to 15m:

keit/helm/values.yaml

Line 23 in 1ede34b

interval: 15m

In this issue I explain why 15m doesn't work for me. 5m would be better. What do you think about it?