Measuring energy on a single machine running multiple concurrent GPU processes (Federated Learning simulation)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 323
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Dear authors,
Thanks for your nice work.
I have been using codecarbon for some time, and I was wondering what is the recommended way to use CodeCarbon to measure energy/emissions on a single machine where multiple GPU processes run in parallel?
Our use case is a federated learning simulation that launches many federated “clients” as separate processes on one GPU.
Right now, we run one EmissionsTracker per process (each simulated client), and this seems to be working reasonably. However, if we track per process, how does CodeCarbon derive the GPU power consumption? Is per-PID attribution supported via NVML, and could that double-count energy cost when multiple processes share a GPU? How should we think about CPU/RAM attribution when multiple processes are active? Are there docs/examples for multi-process, multi-GPU scenarios we should follow?
Thanks in advance! Any pointers to best practices or examples would be greatly appreciated.
Best regards,
Austin Tapp
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named in the issue. Start by reviewing the existing EmissionsTracker and NVML documentation, focusing on process-level GPU, CPU, and RAM attribution for concurrent clients. Done should be clear guidance or examples for single-machine multi-process and multi-GPU tracking without double-counting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100