open-telemetry / open-telemetry/opentelemetry-python
Tests in test_metric_reader_storage.py assert on process-global mock state instead of on the storage under test
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Describe your environment
main (5aa2f8f), opentelemetry-sdk. Not environment specific, but it only surfaces on slow runners (observed on Windows / 3.14t).
What happened?
Several tests in opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py patch the module-level name _ViewInstrumentMatch and then assert on state that lives on the patched class object rather than on the storage under test:
Because @patch rebinds a module attribute, the replacement is visible to the entire process. _ViewInstrumentMatch is constructed by MetricReaderStorage._get_or_init_view_instrument_match, which any MeterProvider in the process can reach:
So if anything else constructs a view-instrument match while one of these tests is running (for example a PeriodicExportingMetricReader ticker thread that a previous test left alive), it mutates the mock these tests are asserting on. The storages themselves stay correctly isolated. The shared state is one level up, on the mock.
There are two distinct vulnerable patterns.
1. Counting via call_args_list. call_args_list records every call to the mock, from anywhere.
test_creates_view_instrument_matches, lines 77, 83, 87, 90test_default_view_enabled, lines 242, 247, 249, 251
An extra construction anywhere makes the count too high. This is the observed failure in #5157:
self.assertEqual(len(MockViewInstrumentMatch.call_args_list), 1)
E AssertionError: 2 != 1
2. A finite side_effect list. test_forwards_calls_to_view_instrument_match, line 94, sets
MockViewInstrumentMatch.side_effect = [
view_instrument_match1,
view_instrument_match2,
view_instrument_match3,
]
An outside construction consumes an entry, so the test's own storage receives the wrong object, or the list is exhausted and StopIteration is raised. Same exposure, different symptom.
What did you expect to see?
These tests should be unaffected by anything happening elsewhere in the process. They are testing one MetricReaderStorage instance, which is private to the test.
Suggested fix
Assert on the object under test instead of on the mock. test_race_concurrent_measurements in the same file already does exactly this and is therefore immune:
self.assertIn(instrument1, storage._instrument_view_instrument_matches)
self.assertEqual(len(storage._instrument_view_instrument_matches[instrument1]), 1)
Applying the same pattern to the two call_args_list tests removes the shared channel entirely. For test_forwards_calls_to_view_instrument_match, a side_effect callable that returns a fresh mock per call (as test_race_concurrent_measurements does with MockFunc) avoids the exhaustion problem.
Relationship to #5157
This is one of two independent problems behind #5157, and it is the one responsible for the actual red test (AssertionError: 2 != 1). Fixing it makes these tests robust regardless of what threads are running.
The other problem is that some test leaks a live PeriodicExportingMetricReader ticker thread, which is what produces the UnboundLocalError thread exception in the same run. That is tracked separately and does not need to be solved to fix this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in opentelemetry-sdk/tests/metrics/test_metric_reader_storage.py, focusing on test_creates_view_instrument_matches, test_default_view_enabled, and test_forwards_calls_to_view_instrument_match. Compare them with test_race_concurrent_measurements and inspect MetricReaderStorage._get_or_init_view_instrument_match in opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/metric_reader_storage.py. Done means the tests assert on their storage and remain unaffected by unrelated constructions or threads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100