mlco2 / mlco2/codecarbon

Feature proposal: `codecarbon doctor` to report measurement quality

Open
#1,350 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
323
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Feature proposal.

The problem

CodeCarbon always produces a number, but it never tells the user how that number was obtained. On a Linux box where /sys/class/powercap/intel-rapl is root-only, the CPU silently falls back to a TDP-times-load model. On an unrecognised CPU model the TDP itself is a generic constant. On macOS without sudo for powermetrics it falls back again. RAM is always modelled, never measured. In every one of those cases the tracker keeps going and writes a confident float to emissions.csv.

The information exists — ResourceTracker picks a backend and logs it, CPU._mode records the mode, CPU._is_generic_tdp records whether the TDP was a real match — but it is only visible at INFO/DEBUG log level, mixed into startup noise, and codecarbon detect (which is where users look) prints a correct hardware list and says nothing about quality. Users find out their measurement was mostly an estimate later, usually from a reviewer.

This is a disclosure problem, not a measurement problem: the fallbacks are reasonable, they are just invisible.

Proposed design

A new CLI command:

codecarbon doctor [--json] [--strict]

It runs the existing hardware detection (EmissionsTracker(save_to_file=False).get_detected_hardware() path) and, for each power component, prints a status, the method, why a better method was not used, and the concrete fix:

CPU     12th Gen Intel(R) Core(TM) i7-1260P
        ESTIMATED - CPU load model over a 28 W TDP
        Why: /sys/class/powercap/intel-rapl exists but its energy counters are
             not readable by this user (permission denied)
        Fix: sudo chmod -R a+r /sys/class/powercap/intel-rapl
             https://docs.codecarbon.io/how-to/enable-rapl/

--json gives a machine-readable version for CI gates and bug reports, --strict exits non-zero when any component is estimated.

The public shape is one dataclass and one function in codecarbon/diagnostics.py:

@dataclass
class ComponentDiagnostic:
    component: str   # "CPU" | "RAM" | "GPU"
    detail: str
    status: str      # "measured" | "estimated" | "unavailable"
    method: str
    reason: str | None
    fix: str | None

def diagnose(hardware) -> list[ComponentDiagnostic]: ...

Why it fits existing extension points

No new measurement code and no new dependency. ResourceTracker already centralises the "which backend won" decision, CPU._mode and CPU._is_generic_tdp already encode the answer, and cpu.is_rapl_available() / powermetrics.is_powermetrics_available() / windows_emi.is_emi_available() already run the checks whose failure reason we want to surface. diagnose() reads the hardware objects the tracker builds; it does not duplicate detection. typer and rich are already required.

Scope boundary

First PR: diagnostics.py, the doctor command, docs. Deliberately out of scope, because they change behaviour for every existing user and deserve their own review:

  • a start-of-run warning from EmissionsTracker.__init__ when a component is estimated;
  • a measurement_quality provenance field on EmissionsData / the CSV;
  • per-backend why_unavailable() methods on each of rapl.py, powermetrics.py, windows_emi.py, gpu_*.py — the first version derives the reason from the existing availability checks instead, which covers the common Linux/macOS cases without touching six modules.

Also explicitly not in scope: doctor running privileged fix commands itself. It prints them.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the existing EmissionsTracker(save_to_file=False).get_detected_hardware() path, ResourceTracker backend selection, and the proposed codecarbon/diagnostics.py interface. Trace CPU._mode, CPU._is_generic_tdp, and the existing availability checks before wiring the doctor command. Done means measured, estimated, or unavailable diagnostics are shown, --json is machine-readable, --strict exits non-zero for estimates, and the docs describe the command.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.