awslabs / awslabs/aws-embedded-metrics-python

Timestamp doesn't represent correct metric time

Open
#53 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
231
Forks
44
PR merge metrics
No merged PRs in 30d

Description

According to the EMF specification each log record's `_aws_` metadata object must have a single "Timestamp" attribute representing the time the metrics are associated with. However when using `aws-embedded-metrics` the timestamp in the metadata is set to the time the function decorated with `@metric_scope` got called. So obviously the timestamp is different to the time `MetricLogger().put_metric()` for individual metrics gets called. This leads to incorrect timestamps as soon as the decorated function emits metrics after more than the minimum resolution shown in CloudWatch Metrics, which I believe is one minute when using EMF.

A single function running longer than one minute is probably pretty common, so I believe this is a real problem. What I'd expect `aws-embedded-metrics` to do is to store the current time together with the metric value when `MetricLogger().put_metric()` is called and create one log record per minute worth of metric values when serializing the data. This should work fine, as metric timestamps submitted to CloudWatch Metrics can be up to two weeks in the past. Maybe there is also the possibility to submit the log records for "finished" resolution intervals asynchronously in the background to get the metric values into CloudWatch Metrics, while the decorated function is still running.

While such a change would unfortunately create more log records, I believe it would more accurately represent the time metrics are associated with.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.