mlco2 / mlco2/codecarbon

Unable to bypass psutil (requesting feature to force constant mode)

Open
#878 7 comments 1 reaction 1 assignee View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
323
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Hello, thank you for your work on CodeCarbon!

I'm currently using CodeCarbon to estimate energy usage in a machine learning setting, where I track short training iterations (e.g forward pass, backward pass, etc).

I'm observing a significant overhead when using CodeCarbon v3.0.2:

  • Full training time without CodeCarbon: 11 seconds
  • Full training time with CodeCarbon v3.0.2: 1 minute 55 seconds

I'm working in an environment without RAPL, PowerGadget, or PowerMetrics. I identified the overhead as coming from repeated calls to psutil.cpu_percent(interval=0.5), which takes 0.5 seconds per call. In contrast, when I use CodeCarbon v2.8.4 (before psutil support) it incurrs negligible overhead (<1s).

I am therefore trying to bypass using psutil (i.e. bypassing CPU Load mode). I attempted to force CodeCarbon into constant mode by uninstalling psutil, expecting it to fall back as it does for unavailable backends like RAPL or PowerGadget. However, this leads to a ModuleNotFoundError due to an unconditional import of psutil in core/cpu.py.

Looking into the source, I saw that CodeCarbon uses the following health check:

is_psutil_available():
    try:
        nice = psutil.cpu_times().nice
        if nice > 0.0001:
            return True
        else:
            logger.debug(
                f"is_psutil_available() : psutil.cpu_times().nice is too small : {nice} !"
            )
            return False
    except Exception as e:
        logger.debug(
            "Not using the psutil interface, an exception occurred while instantiating "
            + f"psutil.cpu_times : {e}",
        )
        return False

This differs for example from is_powergadget_available() which explicitly looks for the Power Gadget executable.

Would it be possible to:

  • Add a parameter (e.g. force_mode_constant=True) to explicitly force constant mode, similar to force_mode_cpu_load
  • Or change the psutil import logic to allow it to fail gracefully when the module is missing, rather than assuming it will always be installed

Alternatively, do you know of a way to make the call to is_psutil_available() fail?

Thanks again for maintaining this project!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.