Flaky: tracker flush and logging-output tests fail intermittently on master
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 323
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Three tests fail intermittently when the whole suite runs, across two files:
tests/test_emissions_tracker_flush.py::test_carbon_tracker_offline_flushtests/test_logging_output.py::test_carbon_tracker_offline_logging_output
The failures are row counts (0 == 1, 2 == 3), so an expected measurement did not land before the assertion. Each test passes in isolation, and it reproduces on master, not just on feature branches.
The most likely cause is how CI runs the suite. .github/workflows/test-package.yml:36 is:
uv run codecarbon monitor -- uv run task test-coverage
so a live tracker wraps the entire test run. It holds /tmp/.codecarbon.lock and writes its own emissions.csv in the repo root for as long as the tests execute, while the tests under it are starting and stopping their own trackers. That is a real interaction and it is not present when you run a single test locally.
Two smaller things that make it worse:
tests/test_logging_output.pybuilds three trackers with nooutput_dirand nooutput_file(lines 48-52, 64-69, and the@track_emissionsdecorator at 81-85), so they all default toemissions.csvin the current working directory. Nothing cleans that up.- Both files use a fixed filename in
tempfile.gettempdir()(emissions-test-TestCarbonTrackerFlush.csv,emissions-test-TestCarbonLoggingOutput.log) with no per-run suffix, andsetUp/tearDownboth delete it.
PeriodicScheduler chaining a threading.Timer per tick may also contribute, since a tracker a test does not stop keeps firing into the next test and tests/conftest.py only clears the hardware probe caches. Unverified. The re-arming timer goes away with #1339 / #1324, so it is worth re-measuring after one of those lands.
Likely fixes: give the logging test an explicit output_dir, make the temp filenames unique per process, and stop asserting on wall-clock-dependent row counts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .github/workflows/test-package.yml:36 and reproduce the full suite using the wrapped test command, then inspect the two failing tests in tests/test_emissions_tracker_flush.py and tests/test_logging_output.py. Compare their tracker lifecycles, fixed temporary filenames, default emissions.csv usage, and the timer behavior described in the issue; done means the full suite passes reliably without wall-clock-dependent row-count failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100