jazzband / jazzband/tablib

Simple csv file can't be parsed correctly because of Sniffer() parameters

Open
#622 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.8k
Forks
617
PR merge metrics
No merged PRs in 30d

Description

Greetings,

First of all, thank you for the work you are doing on this library.

I have the following CSV file, where the delimiter is a colon `:` :
```csv
test:test2
value1:value2
```

I'm now using the `tablib` library to import it:
```python
tmp = open('test.csv', 'r')
tmp2 = tablib.Dataset().load(tmp, format='csv')
```

But the file is wrongly parsed, because the `csv` library did not sniffed the delimiter correctly:
```
test:test2
-------------
value1:value2
```

I saw that you are passing `2048 bytes` of samples to be sniffed (https://github.com/jazzband/tablib/blob/4bbb35a4103fea2e091c15314f0a1c53fc9613cb/src/tablib/formats/_csv.py#L58), I now tested to manually sniff the CSV using only the first line:
```python
dialect = csv.Sniffer().sniff(tmp.readline())
```

The delimiter is now indeed recognized as a colon `:`.

My question is: shouldn't you pass the first line to sniff only (using `readline()`) ? Or maybe fallback on the actual behaviour only if the headers is not present in the file ?

Related issue: #502

Regards

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.