Allow computing emissions and dumping results periodically for other output modes
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 323
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Currently, in the CSV output mode and other output modes results and emissions are only computed and persisted at the end of the run. I would rather get the results dumped to disk in the same way that for the API output mode emissions are computed and partial results persisted periodically:
https://github.com/mlco2/codecarbon/blob/63c6a55fbfa101297e96e601a9eb68bbaa11d2e9/codecarbon/emissions_tracker.py#L700-L713
This way:
- If a run crashes unexpectedly partial results are still saved
- I can visualize the outputs during training
- I can monitor emissions by reading the outputs and preempt runs in case of increased emissions, for example
I think the easiest way would be to allow configuring which output modes should appear in the if statements in the lines above, but ideally one could configure different rates for different outputs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in emissions_tracker.py around lines 700-713 and compare the API output path with the CSV and other output modes. Trace where emissions are computed and partial results are persisted, then determine how configurable output modes and update rates should behave. Done means periodic results are saved for the requested modes, including during interrupted runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100