open-telemetry / open-telemetry/opentelemetry-python

problems with generated exponential histogram

Open
#3,767 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
2.6k
Forks
1k
Avg merge
4d 15h
Merged PRs (30d)
19

Description

Describe your environment

Running with:

  • MacBook Pro Intel
  • MacOS 14.3.1 (intel)
  • Python 3.11.7 (homebrew)

Requirements:

opentelemetry-api>=1.23.0
opentelemetry-sdk>=1.23.0
opentelemetry-proto>=1.23.0
opentelemetry-exporter-otlp>=1.23.0

Steps to reproduce

Run main.py (below) and capture the dump of the exponential histogram.
Source code:

from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader

def main():

    exporter = OTLPMetricExporter()
    meter_provider = MeterProvider([PeriodicExportingMetricReader(exporter)])
    meter = metrics.get_meter(
        name="test-histogram",
        meter_provider=meter_provider,
    )
    histo = meter.create_histogram("test-histogram")
    for i in range(1000):
        for _ in range(i):
            histo.record(i, {"type": "test-histogram"})

    meter_provider.shutdown()

if __name__ == "__main__":
    main()

This is run with the following script:

export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE=DELTA
export OTEL_EXPORTER_OTLP_METRICS_DEFAULT_HISTOGRAM_AGGREGATION=base2_exponential_bucket_histogram

python3 ./main.py

What is the expected behavior?

I expected to see a dump like that generated for the JavaScript SDK in
dump-js.txt.

What is the actual behavior?

The generated exponential histogram in dump-python.txt seems to have the following problems:

  1. There is a negative bucket, even though no negative values are added to the histogram. (This actually causes a downstream vendor to reject the histogram.)
  2. All 160 buckets are provided even though the last 79 are all empty. Other SDKs (e.g. Javascript, dump-js.txt) trim unnecessary buckets, reducing the size of the payload.

Additional context

None.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided main.py with the listed OTLP and exponential-histogram environment variables, then compare dump-python.txt with dump-js.txt. Trace the Python SDK's exponential histogram generation to determine why a negative bucket appears and why trailing empty buckets are retained. Done means the output has no negative bucket for these nonnegative values and omits unnecessary trailing empty buckets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.