Add converter and formatter parameters to csv.reader and csv.writer
Dieses Issue hat noch niemand übernommen.
- Vorherrschende Sprache
- Python
- Sterne
- 77.2k
- Forks
- 35.9k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
Feature or enhancement
Proposal:
The conversion between Python values and CSV fields is hard-coded in both directions. The reader converts unquoted fields with float(), and only in the QUOTE_NONNUMERIC and QUOTE_STRINGS modes. The writer converts every non-string value with str().
This is the common cause of several open issues:
- gh-74232 --
boolis written unquoted asTrue, which cannot be read back. - gh-98485 -- the same for
complex;FractionandIntEnumare affected too, andDecimalsilently round-trips throughfloat. - gh-110852 -- there is no way to write floats with a fixed precision, because preformatted strings are quoted in the
QUOTE_NONNUMERICmode. - gh-85002 -- there is no way to reject values which are neither strings nor numbers.
I propose two parameters, mirroring parse_float in json:
csv.reader(f, converter=None)-- called asconverter(index, field)instead offloat().csv.writer(f, formatter=None)-- called asformatter(index, value)instead ofstr(). It must return a string.
index is the 0-based position of the field in the record. Both default to None, which keeps the current behavior. The hooks only replace the existing calls -- what is not passed to float() or str() now is not passed to them either. Quoting is still decided by the original value.
The index goes first, like in enumerate(). This also makes a wrong one-argument callable fail at once: converter=int raises TypeError on the first field instead of taking the index as the base.
The index makes the hooks per-column, which is what the dtype and converters parameters of pandas.read_csv() are used for:
>>> types = [str, int, Decimal, Fraction]
>>> list(csv.reader(['spam,42,1.10,1/2'], quoting=csv.QUOTE_NONNUMERIC,
... converter=lambda i, field: types[i](field)))
[['spam', 42, Decimal('1.10'), Fraction(1, 2)]]
>>> def money(index, value):
... return format(value, '.2f') if index == 2 else str(value)
>>> csv.writer(sys.stdout, formatter=money).writerow(['a', 1, 0.0, 3.14159])
a,1,0.00,3.14159
gh-85002 no longer needs a parameter of its own -- a strict writer is a formatter which refuses everything except numbers.
I have a working prototype (about 90 lines in Modules/_csv.c).
Open question: should these be parameters of the reader and the writer, or attributes of the dialect? A dialect is a portable description of the file syntax -- it is registered under a global name, sniffed, and copied -- so keeping callables out of it seems better.
Linked PRs
- gh-155099
Beitragsleitfaden
Erste Schritte
- Lies das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreib ins Issue, dass du es übernimmst — das erspart doppelte Arbeit.
- Forke das Repository und arbeite in einem Branch.
- Öffne einen Pull Request, der die Issue-Nummer nennt.
Rechercherichtung
Beginne mit Modules/_csv.c und den im Vorschlag beschriebenen reader/writer-Einstiegspunkten; überprüfe die bestehenden Konvertierungspfade von float() und str() sowie den dort erwähnten Prototyp. Vergleiche die Alternativen für Parameter und Dialekt und nutze anschließend die verlinkte Arbeit gh-155099, um die vereinbarte API und die Abschlusskriterien zu bestimmen.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python
- Bereich
- data
- Issue-Typ
- Feature
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 25/100