mlco2 / mlco2/codecarbon

Carbon-aware scheduling: `codecarbon wait` and a `@carbon_aware` decorator

Open
#1,356 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
323
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Feature proposal.

The problem

CodeCarbon answers "how much did that job cost the atmosphere?" It cannot answer "when should I have run it?"

For jobs that are not latency-sensitive — nightly retrains, hyperparameter sweeps, batch inference, backfills, cron CI — the largest reduction available is usually not making the code faster, it is moving it a few hours. Grid carbon intensity in a typical European or US zone swings by a factor of two to four within a day. Deferring a multi-hour training job from an evening peak to an overnight wind trough is a large cut for a one-line change.

This is the question users ask after their first month with the package: "fine, I know it's 4 kg — now what?"

Proposed design

Three layers, each usable on its own.

1. Forecast retrieval. A new codecarbon/core/intensity_forecast.py exposing IntensityPoint / Forecast dataclasses and get_forecast(geo, token=..., horizon_hours=48) -> Forecast | None. It reuses the existing geolocation (GeoMetadata lat/lon or country_2letter_iso_code) and the already-supported electricitymaps_api_token config key, so existing users need no new credential. When no provider is configured it returns None and every caller degrades to "run now" — a job must never block on a missing credential.

2. Window selection. best_window(forecast, duration, deadline) -> (start, mean_intensity). A sliding mean over the forecast points restricted to windows finishing before the deadline. Pure function, no I/O, trivially testable. It returns "now" whenever nothing better exists, so the run-immediately path is the default rather than a special case.

3. User-facing surfaces.

A codecarbon wait CLI command, alongside codecarbon monitor:

# print the recommendation and exit
$ codecarbon wait --dry-run --deadline 24h --duration 90m
Best start: 2026-08-13 03:00 UTC  (112 gCO2e/kWh, now: 341)  -> saves ~67%

# block until the greenest window, then run under measurement
$ codecarbon wait --deadline 12h --duration 2h -- python train.py

The blocking form delegates to the existing run_and_monitor, so measurement, CSV output and exit-code propagation are unchanged.

And a decorator / context manager in codecarbon/carbon_aware.py:

@carbon_aware(deadline="6h", duration="45m")
def nightly_retrain():
    ...

All options resolve through the existing config hierarchy (.codecarbon.config / CODECARBON_*), so carbon_aware_deadline, carbon_aware_duration and carbon_aware_threshold_g behave like every other key.

Hard dependency: pluggable intensity providers with forecast support

Today the only intensity source that could serve a forecast is Electricity Maps, via codecarbon/core/electricitymaps_api.py. That module is a single hardcoded .../carbon-intensity/latest endpoint whose one function returns emissions for an amount of energy, not intensity — the carbonIntensity value is fetched, converted, and immediately multiplied by energy.kWh. Nothing exposes the raw intensity and nothing looks forward in time.

So this feature realistically depends on the pluggable intensity-provider work: without a provider layer, carbon-aware scheduling can only talk to Electricity Maps, and only for users who already hold a paid token. That is a real but narrow slice of the audience and it caps the value the feature can deliver. Once providers land, get_forecast should become an optional forecast() method on the provider protocol rather than a second HTTP client. If any part of this lands first, the HTTP should be kept in one place so the provider work can absorb it cleanly.

Scope boundary

Explicitly not in scope:

  • A scheduler. This is a time.sleep, not a daemon. It does not fork, daemonise or persist. A process that must survive a reboot belongs to cron, systemd or Airflow, and the docs should say so.
  • A static fallback diurnal profile for users with no provider token. It would broaden reach enormously but risks systematically wrong advice — solar-heavy zones trough at midday, wind-heavy zones trough at night. Better to return "run now" than confident wrong advice.
  • Marginal intensity. Load shifting properly responds to marginal generation; Electricity Maps' average figure is a proxy, not the same thing. The honest framing is "we optimise against average intensity". A marginal-signal mode belongs with the provider work.
  • Region shifting. For cloud jobs the greenest choice is often where, not when. That is a natural sequel and a much larger scope.

Open questions

  • Should EmissionsTracker gain a carbon_aware=True parameter that defers start()? A tracker that silently blocks for six hours is a surprising API; leaning towards keeping deferral explicit at the call site.
  • Recording what a deferral avoided (deferred_seconds, avoided_emissions on EmissionsData) is what makes the feature demonstrable, but should only ship alongside a blocking path — two always-zero CSV columns are a schema change for no reader.
  • A waiting process must not hold codecarbon/lock.py for the duration of the wait.
  • Re-evaluate mid-wait as the forecast updates, or commit to the initial decision? Suggested: re-check coarsely, and only ever move the start time earlier.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with codecarbon/core/electricitymaps_api.py and the pluggable intensity-provider work, then inspect the existing monitor CLI and run_and_monitor path. Define the provider forecast boundary before implementing intensity_forecast.py, best_window, the wait command, and carbon_aware.py. Done means the feature degrades to run-now without credentials, preserves measurement and exit-code behavior, and has tests for forecast selection and deadline handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.