`csv.Sniffer.sniff()` raises `Could not determine delimiter` when the sample contains a field larger than field_size_limit()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
Summary
csv.Sniffer().sniff() raises csv.Error: Could not determine delimiter for a sample whose first row contains a field longer than the current csv.field_size_limit() (131072 by default), even when every following row is consistently delimited.
Reproduction Code
import csv
sample = 'a,"' + 'x' * 200_000 + '",b\n' + 'c,d,e\n' * 50
print(csv.Sniffer().sniff(sample).delimiter)
Actual Behavior
Traceback (most recent call last):
File "repro.py", line 4, in <module>
print(csv.Sniffer().sniff(sample).delimiter)
~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "Lib/csv.py", line 384, in sniff
raise Error("Could not determine delimiter")
_csv.Error: Could not determine delimiter
Expected Behavior
sniff() should return a dialect with `delimiter=','
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
- gh-157689
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the sample from the issue, then inspect Lib/csv.py around Sniffer.sniff(). Check how the field_size_limit affects delimiter detection and verify that the sample returns a dialect whose delimiter is a comma without weakening the limit behavior. The issue's linked PR gh-157689 indicates that work may already be underway.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100