Improve headers detection logic when all columns are of "string" type
Open
fread
improve
low priority
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
Consider a file like this:
```
A,B,C
D,E,F
```
Since all fields (and headers) are of type "string", we presume `header = true`, and print a verbose message that "no better guess is possible". However it **is** possible to have a better guess, by testing whether there are any repeating values within each column in the first 100 rows. For example, if the file is
```
a,b,c,d
a,h,i,j
a,e,c,
z,e,n,m
```
then it is more reasonable to assume that the first row is data, since it would be very uncommon for a column's header to have the same value as one of the cells.
An example file: https://s3.amazonaws.com/h2o-public-test-data/fread/Mushroom.gz
Contributor guide
Assessment
This issue has not been assessed yet.