Multi-variable `melt(..., na.rm=T)` removes all rows with at least one NA

Open

#3,896 opened on Sep 19, 2019 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 3/5
Estimated time: 1-2 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: r
Domain: data

Research direction

Start by reproducing the multi-variable case with melt.data.table shown in the issue, comparing na.rm=T and na.rm=F. Trace how na.rm handles rows across multiple measure variables, then verify that only rows where all melted values are NA are removed and that the demonstrated output is retained.

Written by the indexing model from the issue text.

Description

feature request reshape

I think the behavior of na.rm=T with multiple melting variables should follow the AND logic not OR logic as it seems to be the case currently. The current behavior is not at all useful and I believe it's a bug.

# Minimal reproducible example

library(data.table)
(tt <- data.table(a = rep_len(c("x", "y", NA), 10),
                  b = rep.int(c(2, NA), 5),
                  id = 1:10))
#>        a  b id
#>  1:    x  2  1
#>  2:    y NA  2
#>  3: <NA>  2  3
#>  4:    x NA  4
#>  5:    y  2  5
#>  6: <NA> NA  6
#>  7:    x  2  7
#>  8:    y NA  8
#>  9: <NA>  2  9
#> 10:    x NA 10

melt.data.table(tt, "id", measure.vars = list(a = 1, b = 2), na.rm = T)
#>    id variable a b
#> 1:  1        1 x 2
#> 2:  5        1 y 2
#> 3:  7        1 x 2

Expected to remove only those rows for which all rows contain NAs, e.g row 6 in the melted data:

melt.data.table(tt, "id", measure.vars = list(a = 1, b = 2), na.rm = F)
#>     id variable    a  b
#>  1:  1        1    x  2
#>  2:  2        1    y NA
#>  3:  3        1 <NA>  2
#>  4:  4        1    x NA
#>  5:  5        1    y  2
#>  6:  6        1 <NA> NA
#>  7:  7        1    x  2
#>  8:  8        1    y NA
#>  9:  9        1 <NA>  2
#> 10: 10        1    x NA

^{Created on 2019-09-19 by the reprex package (v0.3.0)}

# Output of sessionInfo()

sessionInfo()
R Under development (unstable) (2019-08-06 r76924)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /home/vspinu/bin/R-latest/lib/libRblas.so
LAPACK: /home/vspinu/bin/R-latest/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] VSR_0.0.0.9000       forcats_0.4.0        ggplot2_3.2.1        purrr_0.3.2          htmltidy_0.5.0       xml2_1.2.2           lubridate_1.7.4.9000 data.table_1.12.2    glue_1.3.1           dplyr_0.8.3          DBI_1.0.0           
[12] odbc_1.1.6           lightgbm_2.2.4       R6_2.4.0             jsonlite_1.6         magrittr_1.5         rlang_0.4.0         

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2       compiler_3.7.0   pillar_1.4.2     tools_3.7.0      zeallot_0.1.0    digest_0.6.20    bit_1.1-14       evaluate_0.14    tibble_2.1.3     gtable_0.3.0     pkgconfig_2.0.2  reprex_0.3.0     xfun_0.9        
[14] knitr_1.24       withr_2.1.2      fs_1.3.1         vctrs_0.2.0      htmlwidgets_1.3  hms_0.5.0        bit64_0.9-7      grid_3.7.0       tidyselect_0.2.5 processx_3.4.1   XML_3.98-1.20    rmarkdown_1.15   callr_3.3.1     
[27] whisker_0.3-2    clipr_0.7.0      blob_1.2.0       ps_1.3.0         backports_1.1.4  scales_1.0.0     htmltools_0.3.6  assertthat_0.2.1 colorspace_1.4-1 lazyeval_0.2.2   munsell_0.5.0    crayon_1.3.4

Dominant language: R
Stars: 3.9k
Forks: 1.1k
Avg merge: 14h 4m
Merged PRs (30d): 4

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

Support zstd-compressed package archives: CRAN's current macOS binaries fail with "unknown archive type" Open

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

r-lib/pkgdepends#485 · 3 comments · Opened Sep 20, 2026
entrez_fetch(parsed = TRUE) checks rettype "gpc" but parse_response() handles "gbc" Open

Difficulty 1/5 Under an hour Newbie friendliness 92/100

ropensci/rentrez#228 · Opened Sep 20, 2026
Nobody is told to clone the repository, and data/ means two different directories Open

beginners blocker

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

naturalis/DABDGE#17 · 1 comment · Opened Sep 19, 2026
enviPathR Open

Build Error Build OK Build Warning policies-accepted pre-review precheck-passed

Difficulty 1/5 Under an hour Newbie friendliness 84/100

Bioconductor/BiocContributions#207 · 6 comments · Opened Sep 19, 2026
Consider adding stacked and dodged bar plots Open

Difficulty 2/5 1-3 hours Newbie friendliness 74/100

datacarpentry/semester-biology#1255 · Opened Sep 19, 2026

More R issues