open-telemetry / open-telemetry/opentelemetry-python
PeriodicMetricExporter does not export periodically for either Delta or Cumulative temporality ExponentialHistogram
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Describe your environment
OS: Centos 7
Python version: 3.11.1
SDK version: 1.25.0
API version: 1.25.0
What happened?
I have a low-throughput service and have implemented my own Histogram to measure the duration of an operation. This uses a PeriodicMetricExporter which is configured to export every 60 seconds. When I use Delta temporality, I get an export within 60 seconds of the last histogram recording. I then have no exports until the next observation is made.
When changing to Cumulative temporality I have the exact same experience. For comparison purposes, when using the Golang SDK I get the same behaviour as Python's Delta but, when using Cumulative, I get a metronomic export every 60 seconds regardless of any observations being recorded..
Should the Python PeriodicMetricExporter export every 60 seconds or not, and how does temporality setting play into it? At the moment it causes me a problem because my service has low throughput. My observability backend (New Relic) uses the StartTimeUnixNano value which can be hours in the past. A constant periodic export would help to correct this as the StartTimeUnixNano would be at most 60 seconds in the past.
Steps to Reproduce
metrics_temporality = {
Histogram: AggregationTemporality.DELTA
}
otlp_metric_exporter = OTLPMetricExporter(endpoint=grpc_collector_address, insecure=True,
preferred_temporality=metrics_temporality, timeout=30_000)
metric_reader = PeriodicExportingMetricReader(otlp_metric_exporter,
export_interval_millis=60_000))
meter_provider = MeterProvider(metric_readers=[metric_reader], resource=_create_resources(),
views=[View(instrument_type=Histogram,
aggregation=ExponentialBucketHistogramAggregation())])
metrics.set_meter_provider(meter_provider)
meter = metrics.get_meter(__name__)
h = meter.create_histogram(
name='my_histogram', unit='s', description='just my histogram'
)
h.record(1.0)
time.sleep(120)
h.record(2.0)
Expected Result
I expected to see periodic metric exports for the histogram every 60 seconds, regardless of chosen temporality
Actual Result
A metric export within 60 seconds of the last observation.
Additional context
N/A
Would you like to implement a fix?
None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with PeriodicExportingMetricReader and the ExponentialBucketHistogramAggregation setup shown in the reproduction, then compare exports for Delta and Cumulative temporality with no observations between recordings. Check the OTLPMetricExporter path and relevant metric-reader behavior. Done means the export cadence and StartTimeUnixNano behavior are defined and covered for both temporality settings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100