on_csv_write="update" crashes on second write when a column is empty in all existing rows
@princebekui is already working on this.
Since Sep 17, 2026.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 323
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 12
Description
Problem
With on_csv_write="update", the second write for a given run_id crashes when any column is empty in every existing CSV row. This is easy to hit in practice: an OfflineEmissionsTracker leaves longitude/latitude empty, and gpu_count/gpu_model are empty on CPU-only machines. Any run that writes twice — a flush() followed by stop(), or a tracker with measure_power_secs scheduling — crashes on the second write.
Reproduction
import tempfile
from codecarbon.output_methods.emissions_data import EmissionsData
from codecarbon.output_methods.file import FileOutput
d = EmissionsData(
timestamp="2023-01-01T00:00:00", project_name="p", run_id="r", experiment_id="e",
duration=10, emissions=0.5, emissions_rate=0.05, cpu_power=20, gpu_power=0,
ram_power=5, cpu_energy=200, gpu_energy=0, ram_energy=50, energy_consumed=250,
water_consumed=0.1, country_name="Testland", country_iso_code="TS", region="R",
cloud_provider="", cloud_region="", os="TestOS", python_version="3.8",
codecarbon_version="2.0", cpu_count=4, cpu_model="CPU", gpu_count=None,
gpu_model=None, longitude="", latitude="", ram_total_size=16, tracking_mode="machine",
)
f = FileOutput("test.csv", tempfile.mkdtemp(), on_csv_write="update")
f.out(d, None)
f.out(d, None) # raises
Root cause
codecarbon/output_methods/file.py:122, in the branch handling exactly one existing row with the current run_id:
update_values = {}
for col, val in dict(total.values).items():
update_values[col] = df[col].dtype.type(val)
df[col].dtype.type(val) coerces the incoming value to whatever dtype pandas inferred when reading the existing CSV back. A column that is empty in every existing row is read back as float64, so this evaluates numpy.float64("") or numpy.float64(None).
Expected vs actual
Expected: the existing row for the run is replaced by the new values, empty columns included.
Actual:
File "codecarbon/output_methods/file.py", line 122, in out
update_values[col] = df[col].dtype.type(val)
ValueError: could not convert string to float: ''
(None instead of "" gives TypeError: float() argument must be a string or a real number, not 'NoneType'.)
The coercion serves no purpose here — pandas can assign the values directly, or the row can simply be rebuilt.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.