`csv` does not round-trip for `complex` numbers

Open
#98,485 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python
Domain
data

Research direction

Start by reproducing the reported Python csv examples for QUOTE_ALL and QUOTE_NONNUMERIC, then review the csv documentation and tests mentioned in the issue. Determine the intended behavior for complex numbers, implement the agreed direction, and add matching documentation or test coverage so round-tripping and conversion behavior are explicit.

Written by the indexing model from the issue text.

Description

stdlib type-bug

Reproduction depends on the quoting.

QUOTE_ALL

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_ALL)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_ALL)
    print(list(reader))  # [['1'], ['', ''], ['1'], ['j'], []]

In this case, we don't have our data back, but at least it does not raise.

QUOTE_NONNUMERIC

Qouting docs:

No automatic data type conversion is performed unless the QUOTE_NONNUMERIC format option is specified (in which case unquoted fields are transformed into floats)

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_NONNUMERIC)
    print(list(reader))

# Traceback (most recent call last):
#   File "/Users/sobolev/Desktop/cpython/ex.py", line 11, in <module>
#     print(list(reader))
#           ^^^^^^^^^^^^
# ValueError: could not convert string to float: 'j'

In this case, it raises an error, while trying to convert 1j to float.

Current docs / tests

I cannot find any mentions of complex numbers in tests or docs for csv.

Solutions?

  1. We can say that it works as expected: with this strange QUOTE_ALL formatting and QUOTE_NONNUMERIC exception. Add a test case for it and forget about it
  2. We can try to quote complex as string: in this case it will be treated as "1j". I think it is much better, because it will allow users to convert this value to complex manually
  3. Forbid writting complex numbers. However, I don't think it is a path we should go
Dominant language
Python
Stars
77.2k
Forks
36k
Avg merge
1d 9h
Merged PRs (30d)
558

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python/cpython

All issues in python/cpython

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.