fread(logicalYN=TRUE) could avoid detecting 'y'/'n' in header detection?
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
Research direction
Start with the tests referenced from pull request #4564 and trace fread’s header detection and logicalYN type-detection paths. Compare how the first row is classified with later data rows, including headers containing y or n and header=FALSE inputs. Done means false positives are covered by tests and logicalYN can safely be enabled by default without breaking existing cases.
Written by the indexing model from the issue text.
Description
Follow-up to #4564. There we set logicalYN=FALSE by default for back-compatibility because currently auto-detection of header has a bad interaction with logicalYN -- fread finds y in the header row and thinks "that's non-character data, namely, TRUE", thereby concluding header=FALSE. See the tests in that PR.
Ideally we can disable this parser for the header, but I don't think we have any logic for subsetting the valid parsers for the header only.
We'd also want to think through if doing so will create any undesirable edge cases. The only thing I can think of is something pathological like a file where header=FALSE that consists only of columns named y or n, then we might incorrectly detect header=TRUE based on the string-only data in the first row. AFAICT any realistic example where there are other columns would do fine. But perhaps we can also revisit the header detection logic -- I might think something like "do type detection on row 1; then do type detection on samples of rows 2...N, and compare the inferred types" would be fine.
Anyway, if we can remove these false positives of detecting y --> TRUE, I think we could switch to logicalYN=TRUE by default.
- Dominant language
- R
- Stars
- 3.9k
- Forks
- 1.1k
- Avg merge
- 14h 4m
- Merged PRs (30d)
- 4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Rdatatable/data.table
-
as.data.table() recurses without end on a survival::Surv object (or any data.frame carrying one) Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Rdatatable/data.table#7887 ·
-
consistency tests
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#7853 · 3 comments ·
-
internals
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#6938 · 1 comment ·
-
encoding fread
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#5179 · 8 comments ·
-
documentation programming
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#3199 · 3 comments ·
All issues in Rdatatable/data.table
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
beginners blocker
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enviPathR OpenBuild Error Build OK Build Warning policies-accepted pre-review precheck-passed
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Bioconductor/BiocContributions#207 · 6 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
datacarpentry/semester-biology#1255 ·