mlco2 / mlco2/codecarbon

on_csv_write="update" crashes on second write when a column is empty in all existing rows

Open
#1,367 4 comments 0 reactions 1 assignee View on GitHub

@princebekui is already working on this.

Since Sep 17, 2026.

Dominant language
Python
Stars
1.9k
Forks
323
Avg merge
1d 12h
Merged PRs (30d)
12

Description

Problem

With on_csv_write="update", the second write for a given run_id crashes when any column is empty in every existing CSV row. This is easy to hit in practice: an OfflineEmissionsTracker leaves longitude/latitude empty, and gpu_count/gpu_model are empty on CPU-only machines. Any run that writes twice — a flush() followed by stop(), or a tracker with measure_power_secs scheduling — crashes on the second write.

Reproduction

import tempfile
from codecarbon.output_methods.emissions_data import EmissionsData
from codecarbon.output_methods.file import FileOutput

d = EmissionsData(
    timestamp="2023-01-01T00:00:00", project_name="p", run_id="r", experiment_id="e",
    duration=10, emissions=0.5, emissions_rate=0.05, cpu_power=20, gpu_power=0,
    ram_power=5, cpu_energy=200, gpu_energy=0, ram_energy=50, energy_consumed=250,
    water_consumed=0.1, country_name="Testland", country_iso_code="TS", region="R",
    cloud_provider="", cloud_region="", os="TestOS", python_version="3.8",
    codecarbon_version="2.0", cpu_count=4, cpu_model="CPU", gpu_count=None,
    gpu_model=None, longitude="", latitude="", ram_total_size=16, tracking_mode="machine",
)

f = FileOutput("test.csv", tempfile.mkdtemp(), on_csv_write="update")
f.out(d, None)
f.out(d, None)  # raises

Root cause

codecarbon/output_methods/file.py:122, in the branch handling exactly one existing row with the current run_id:

update_values = {}
for col, val in dict(total.values).items():
    update_values[col] = df[col].dtype.type(val)

df[col].dtype.type(val) coerces the incoming value to whatever dtype pandas inferred when reading the existing CSV back. A column that is empty in every existing row is read back as float64, so this evaluates numpy.float64("") or numpy.float64(None).

Expected vs actual

Expected: the existing row for the run is replaced by the new values, empty columns included.

Actual:

File "codecarbon/output_methods/file.py", line 122, in out
    update_values[col] = df[col].dtype.type(val)
ValueError: could not convert string to float: ''

(None instead of "" gives TypeError: float() argument must be a string or a real number, not 'NoneType'.)

The coercion serves no purpose here — pandas can assign the values directly, or the row can simply be rebuilt.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.