open-telemetry / open-telemetry/opentelemetry-ruby
Asynchronous instruments produce incorrect cumulative and delta output
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 606
- Forks
- 301
- Avg merge
- 3d 19h
- Merged PRs (30d)
- 42
Description
Asynchronous instruments (ObservableCounter, ObservableUpDownCounter) produce incorrect output across multiple collections. The callback reports the absolute cumulative value at collection time, but the SDK feeds that value into Sum#update, which does ndp.value += increment (the same increment path synchronous instruments use) and the async stream never differences successive observations. The result is wrong under both temporalities.
With a callback that steadily returns 1000, across three collections:
| Temporality | Expected | Actual |
|---|---|---|
| Cumulative | 1000, 1000, 1000 |
1000, 2000, 3000 (absolute value is +='d across collects) |
| Delta | 1000, 0, 0 (first obs is +1000 from start; no change after) |
1000, 1000, 1000 (store is cleared each collect, so the raw absolute is re-emitted instead of the difference. A delta consumer that reconstructs a total, e.g. Prometheus total += delta, sums these back to 1000, 2000, 3000) |
It's clearest with a monotonic ObservableCounter reporting a running total (e.g. total requests served): a callback steady at 5000 should export 5000, 5000, 5000 (cumulative) or 5000, 0, 0 (delta), but currently gives 5000, 10000, 15000.
Per the Metrics SDK spec, an asynchronous instrument's callback reports the cumulative absolute value; the SDK should export it directly as cumulative, or difference successive observations to derive delta. The current instruments do neither.
Root cause:
AsynchronousMetricStream#invoke_callbackpasses the callback's return value straight into@default_aggregation.update(value, ...). see:asynchronous_metric_stream.rb:62.Sum#updatetreats its argument as an increment:ndp.value += increment. see:sum.rb:92.
This is correct for synchronous instruments (each .add(x) genuinely is an increment) but incorrect for asynchronous ones (each callback is an absolute total). It has gone unnoticed because the async tests (observable_counter_test.rb, observable_up_down_counter_test.rb) each pull only once, so the across-collect behavior is never exercised.
Share details about your runtime
Operating system details: macOS (Darwin 25)
RUBY_ENGINE: "ruby"
RUBY_VERSION: "3.4.9"
RUBY_DESCRIPTION: "ruby 3.4.9 (2026-03-11 revision 76cca827ab) +PRISM [arm64-darwin25]"
opentelemetry-metrics-sdk: 0.16.0
opentelemetry-metrics-api: 0.4.0
Share a simplified reproduction if possible
require 'bundler/inline'
gemfile(true) do
source 'https://rubygems.org'
gem 'opentelemetry-metrics-api'
gem 'opentelemetry-metrics-sdk'
end
require 'opentelemetry-metrics-sdk'
# Minimal exporter that records the value of every data point it sees.
class Capture
attr_reader :rows
def initialize = @rows = []
def export(metrics, timeout: nil)
metrics.each { |m| m.data_points.each { |dp| @rows << dp.value } }
OpenTelemetry::SDK::Metrics::Export::SUCCESS
end
def force_flush(timeout: nil) = OpenTelemetry::SDK::Metrics::Export::SUCCESS
def shutdown(timeout: nil) = OpenTelemetry::SDK::Metrics::Export::SUCCESS
end
OpenTelemetry::SDK.configure
cap = Capture.new
reader = OpenTelemetry::SDK::Metrics::Export::PeriodicMetricReader.new(exporter: cap)
OpenTelemetry.meter_provider.add_metric_reader(reader)
# Callback reports a constant absolute value of 1000 on every collection.
OpenTelemetry.meter_provider.meter('repro')
.create_observable_up_down_counter('test.oudc', callback: -> { 1000 }, unit: 'By')
3.times { reader.force_flush }
p cap.rows
# Expected: [1000, 1000, 1000] - the observed absolute value each collection.
# Actual: [1000, 2000, 3000] - the value accumulates across collections.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with metrics_sdk/lib/opentelemetry/sdk/metrics/state/asynchronous_metric_stream.rb:62 and aggregation/sum.rb:92, then compare the existing observable_counter_test.rb and observable_up_down_counter_test.rb coverage. Exercise three collections under cumulative and delta temporalities; done means constant absolute callback values produce 1000, 1000, 1000 cumulatively and 1000, 0, 0 as deltas, with regression tests covering both instruments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100