Another seemingly bug in dealing with NAs

Open
#5,273 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Start by running the minimal fread example from the issue and compare its output with the version where quoted NA values are replaced by XX. Trace fread's handling of NA strings, quoted values, and empty fields, then verify that a regression test preserves the expected distinction between empty fields, unquoted NA, and quoted "NA".

Written by the indexing model from the issue text.

Description

fread

Too much requirement readings... I think I did my best :)
NA seems to cause all sorts of problem...

Here is my Minimal Example.

txt = r"(
NA,"NA","",
"NA",,"",NA
"",NA,,"NA"
)"
fread(txt) # whether header=TRUE or FALSE
#    V1 V2 V3 V4
# 1: NA NA NA NA
# 2: NA NA NA NA
# 3: NA NA NA NA

Here is what it should do(it is just that "NA" is changed to "XX")

txt = r"(
NA,"XX","",
"XX",,"",NA
"",NA,,"XX"
)"
fread(txt, header=FALSE)
#      V1   V2 V3   V4
# 1: <NA>   XX NA     
# 2:   XX      NA <NA>
# 3:      <NA> NA   XX

except for one thing that V3 are all NAs... where is ""?

It doesnot matter whether or not you set na.strings = 'NA'

> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19043)

Matrix products: default

locale:
[1] LC_COLLATE=Korean_Korea.949  LC_CTYPE=Korean_Korea.949    LC_MONETARY=Korean_Korea.949
[4] LC_NUMERIC=C                 LC_TIME=Korean_Korea.949    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.14.2 dplyr_1.0.7      

loaded via a namespace (and not attached):
 [1] fansi_0.5.0      assertthat_0.2.1 utf8_1.2.2       crayon_1.4.2     R6_2.5.1        
 [6] DBI_1.1.1        lifecycle_1.0.1  magrittr_2.0.1   pillar_1.6.4     cli_3.1.0       
[11] rlang_0.4.12     rstudioapi_0.13  vctrs_0.3.8      generics_0.1.1   ellipsis_0.3.2  
[16] tools_4.1.0      glue_1.4.2       purrr_0.3.4      compiler_4.1.0   pkgconfig_2.0.3 
[21] tidyselect_1.1.1 tibble_3.1.5
Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.