Memory leak in CW agent prometheus 1.247348.0b251302

After upgrading CW agent Prometheus from 1.247347.5b250583 to 1.247348.0b251302 the pod started getting killed by Kubernetes (OOMKilled).
Memory limit is set to 2000m. Tried increasing the limit up to 8000m to no avail.
Downgrading to 1.247347.5b250583 fixes the issue (with 2000m limit).
We run the agent in EKS 1.19.

We are experiencing this in a couple environments, each running over 120 pods (including those of daemonsets). Environments where this is not an issue have ~30-50 pods running.
Last messages in the container logs of the killed pods aren't consistent
one instance:
```
...
2021-09-02T15:49:55Z D! [outputs.cloudwatchlogs] Wrote batch of 179 metrics in 1.201294006s
2021-09-02T15:49:55Z D! [outputs.cloudwatchlogs] Buffer fullness: 0 / 10000 metrics
2021-09-02T15:49:55Z D! [outputs.cloudwatchlogs] Pusher published 8 log events to group: /aws/containerinsights/redacted/prometheus stream: redacted with size 4 KB in 110.502908ms.
2021-09-02T15:49:56Z D! [outputs.cloudwatchlogs] Buffer fullness: 0 / 10000 metrics
```
another instance (same cluster):
```
...
2021-09-02T15:57:29Z D! Drop metric with NaN or Inf value: &{map[app:redacted app_kubernetes_io_instance:redacted app_kubernetes_io_name:redacted application:redacted client_id:producer-1 instance:10.1.2.3:15020 istio_io_rev:default job:kubernetes-pods kafka_version:2.6.0 kubernetes_namespace:redacted kubernetes_pod_name:redacted-6dfd7cd497-qr2hb node_id:node--1 pod_template_hash:6dfd7cd497 prom_metric_type:gauge security_istio_io_tlsMode:istio service_istio_io_canonical_name:redacted service_istio_io_canonical_revision:0.1.0 spring_id:redacted.items.producer.producer-1 version:0.1.0] kafka_producer_node_response_rate kafka_producer_node_response_rate kubernetes-pods 10.1.2.3 NaN gauge 1630598180331}
2021-09-02T15:57:29Z D! Drop metric with NaN or Inf value: &{map[app:redacted app_kubernetes_io_instance:redacted app_kubernetes_io_name:redacted application:redacted client_id:producer-2 instance:10.1.2.3:15020 istio_io_rev:default job:kubernetes-pods kafka_version:2.6.0 kubernetes_namespace:redacted kubernetes_pod_name:redacted-6dfd7cd497-qr2hb node_id:node--1 pod_template_hash:6dfd7cd497 prom_metric_type:gauge security_istio_io_tlsMode:istio service_istio_io_canonical_name:redacted service_istio_io_canonical_revision:0.1.0 spring_id:redacted.items.producer.producer-2 version:0.1.0] kafka_producer_node_response_rate kafka_producer_node_response_rate kubernetes-pods 10.1.2.3 NaN gauge 1630598180331}
2021-09-02T15:57:29Z D! Drop metric with NaN or Inf value: &{map[app:redacted app_kubernetes_io_instance:redacted app_kubernetes_io_name:redacted application:redacted client_id:consumer-redacted-group-1 instance:10.1.2.3:15020 istio_io_rev:default job:kubernetes-pods kafka_version:2.6.0 kubernetes_namespace:redacted kubernetes_pod_name:redacted-6dfd7cd497-qr2hb pod_template_hash:6dfd7cd497 prom_metric_type:gauge security_istio_io_tlsMode:istio service_istio_io_canonical_name:redacted service_istio_io_canonical_revision:0.1.0 spring_id:redacted.items.consumer.consumer-redacted-group-1 version:0.1.0] kafka_consumer_last_poll_seconds_ago kafka_consumer_last_poll_seconds_ago kubernetes-pods 10.1.2.3 NaN gauge 1630598180331}
2021-09-02T15:57:29Z D! Drop metric with NaN or Inf value: &{map[app:redacted app_kubernetes_io_instance:redacted app_kubernetes_io_name:redacted application:redacted client_id:consumer-redacted-group-3 instance:10.1.2.3:15020 istio_io_rev:default job:kubernetes-pods kafka_version:2.6.0 kubernetes_namespace:redacted kubernetes_pod_name:redacted-6dfd7cd497-qr2hb pod_template_hash:6dfd7cd497 prom_metric_type:gauge security_istio_io_tlsMode:istio service_istio_io_canonical_name:redacted service_istio_io_canonical_revision:0.1.0 spring_id:redacted.items.consumer.consumer-redacted-group-3 version:0.1.0] kafka_consumer_last_poll_seconds_ago kafka_consumer_last_poll_seconds_ago kubernetes-pods 10.1.2.3 NaN gauge 1630598180331}
```

Let me know if you need any other information/logs that will help in troubleshooting.