mlco2 / mlco2/codecarbon

PeriodicScheduler re-arms itself after stop(), so measurements can outlive tracker.stop()

Open
#1,308 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
323
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Problem

PeriodicScheduler can keep running (and can run its payload) after stop() has returned. In the tracker this means _measure_power_and_energy may execute after EmissionsTracker.stop() has already written the final row and exited the output handlers — producing an extra measurement, a post-run live_out push (Prometheus/API handlers), or an exception logged after "Done!".

Reproduction

import threading, time
from codecarbon.external.scheduler import PeriodicScheduler

calls = []
s = PeriodicScheduler(0.05, lambda: calls.append(1))
gate, orig = threading.Event(), s.start
# widen the (real, but narrow) window between the timer firing and _run reaching start()
s.start = lambda from_run=False: (from_run and gate.wait(2), orig(from_run=from_run))[1]
orig(from_run=False)
time.sleep(0.2)   # timer fired, _run is blocked before start()
s.stop()
gate.set()
time.sleep(0.4)
print(s._stopped, len(calls))   # -> False, and the count keeps climbing forever

Root cause

codecarbon/external/scheduler.py:26-42. _run() calls start(from_run=True), and the from_run flag deliberately bypasses the _stopped check, so a fired timer always arms its successor:

if from_run or self._stopped:
    self._stopped = False
    self._timer = Timer(self.interval, self._run)

If stop() lands between the timer firing and _run acquiring the lock, stop() cancels an already-fired (uncancellable) timer and flips _stopped = True; _run then sets it back to False and arms a timer nobody holds a reference to — the scheduler becomes unstoppable. Separately, stop() reads _stopped outside the lock (scheduler.py:48), and _run never re-checks the flag before invoking the payload, so the payload runs to completion after stop() returned. EmissionsTracker.stop() sets self._scheduler = None immediately after (emissions_tracker.py:909-914), so that late payload mutates tracker state whose handlers have already been exit()ed (emissions_tracker.py:937-938).

Expected vs actual

Expected: after stop() returns, no further payload invocations and no re-arming.
Actual: the reschedule path is not serialized against the stop path over _stopped, so both a stray timer and a late payload invocation can survive stop().

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read codecarbon/external/scheduler.py around lines 26-48 first, then inspect emissions_tracker.py lines 909-938 and run the reproduction in the issue. Done means stop() prevents further payload invocations and re-arming, without measurements or handler activity after tracker shutdown.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.