open-telemetry / open-telemetry/opentelemetry-python

Tests in test_metric_reader_storage.py assert on process-global mock state instead of on the storage under test

Open Beginner friendly
#5,638 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.6k
Forks
1k
Avg merge
4d 15h
Merged PRs (30d)
19

Description

Describe your environment

main (5aa2f8f), opentelemetry-sdk. Not environment specific, but it only surfaces on slow runners (observed on Windows / 3.14t).

What happened?

Several tests in opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py patch the module-level name _ViewInstrumentMatch and then assert on state that lives on the patched class object rather than on the storage under test:

https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py#L54

Because @patch rebinds a module attribute, the replacement is visible to the entire process. _ViewInstrumentMatch is constructed by MetricReaderStorage._get_or_init_view_instrument_match, which any MeterProvider in the process can reach:

https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/metric_reader_storage.py#L65-L93

So if anything else constructs a view-instrument match while one of these tests is running (for example a PeriodicExportingMetricReader ticker thread that a previous test left alive), it mutates the mock these tests are asserting on. The storages themselves stay correctly isolated. The shared state is one level up, on the mock.

There are two distinct vulnerable patterns.

1. Counting via call_args_list. call_args_list records every call to the mock, from anywhere.

  • test_creates_view_instrument_matches, lines 77, 83, 87, 90
  • test_default_view_enabled, lines 242, 247, 249, 251

An extra construction anywhere makes the count too high. This is the observed failure in #5157:

self.assertEqual(len(MockViewInstrumentMatch.call_args_list), 1)
E       AssertionError: 2 != 1

2. A finite side_effect list. test_forwards_calls_to_view_instrument_match, line 94, sets

MockViewInstrumentMatch.side_effect = [
    view_instrument_match1,
    view_instrument_match2,
    view_instrument_match3,
]

An outside construction consumes an entry, so the test's own storage receives the wrong object, or the list is exhausted and StopIteration is raised. Same exposure, different symptom.

What did you expect to see?

These tests should be unaffected by anything happening elsewhere in the process. They are testing one MetricReaderStorage instance, which is private to the test.

Suggested fix

Assert on the object under test instead of on the mock. test_race_concurrent_measurements in the same file already does exactly this and is therefore immune:

https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py#L192-L193

self.assertIn(instrument1, storage._instrument_view_instrument_matches)
self.assertEqual(len(storage._instrument_view_instrument_matches[instrument1]), 1)

Applying the same pattern to the two call_args_list tests removes the shared channel entirely. For test_forwards_calls_to_view_instrument_match, a side_effect callable that returns a fresh mock per call (as test_race_concurrent_measurements does with MockFunc) avoids the exhaustion problem.

Relationship to #5157

This is one of two independent problems behind #5157, and it is the one responsible for the actual red test (AssertionError: 2 != 1). Fixing it makes these tests robust regardless of what threads are running.

The other problem is that some test leaks a live PeriodicExportingMetricReader ticker thread, which is what produces the UnboundLocalError thread exception in the same run. That is tracked separately and does not need to be solved to fix this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py, focusing on test_creates_view_instrument_matches, test_default_view_enabled, and test_forwards_calls_to_view_instrument_match. Compare them with test_race_concurrent_measurements and inspect MetricReaderStorage._get_or_init_view_instrument_match in opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/metric_reader_storage.py. Done means the tests assert on their storage and remain unaffected by unrelated constructions or threads.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.