setDT should check if any column is POSIXlt

Open
#4,800 17 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Start with the minimal POSIXlt example and the setDT(x) entry point, then compare its behavior with as.data.table.list(), which already emits a conversion warning. Determine the intended handling for POSIXlt columns and add coverage showing that setDT() no longer leaves an object that fails in head(); the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Description

consistency

I was doing setDT() of a dataframe (or tibble) with a variable POSIXlt. It seems not to be any problem. Even I can do str() of that object. It does not show anything strange. Then errors started to appear when I tried to manipulate the object or simply do head().

As I have found the problem, I would ask/suggest why do not produce the error when running setDT or, otherwise show a warning?, which would tell about the problem that there are POSIXlt variables and they are not supported. Possibly these variables could be removed or coerced to POSIXct or another better format for data.table and the warning would tell about that.

Update: Minimal example with session info:

> library(tibble)
> library(data.table)
> now <- as.POSIXlt(Sys.time())
> x <- as.data.frame(tibble(now))
> mdt <- data.table(id=1:3, d=strptime(c("06:02:36", "06:02:48", "07:03:12"), "%H:%M:%S"))
Warning message:
In as.data.table.list(x, keep.rownames = keep.rownames, check.names = check.names,  :
  POSIXlt column type detected and converted to POSIXct. We do not recommend use of POSIXlt at all because it uses 40 bytes to store one date.
> # previous is fine; but next...
> setDT(x)
> head(x)
Error in dimnames(x) <- dn : 
  length of 'dimnames' [1] not equal to array extent
> str(x)
Classes ‘data.table’ and 'data.frame':	11 obs. of  1 variable:
 $ now: POSIXlt, format: "2020-11-04 10:24:47"
 - attr(*, ".internal.selfref")=<externalptr> 
> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.7 LTS

Matrix products: default
BLAS:   /usr/lib/atlas-base/atlas/libblas.so.3.0
LAPACK: /usr/lib/atlas-base/atlas/liblapack.so.3.0

locale:
 [1] LC_CTYPE=C.UTF-8       LC_NUMERIC=C           LC_TIME=C.UTF-8        LC_COLLATE=C.UTF-8    
 [5] LC_MONETARY=C.UTF-8    LC_MESSAGES=C.UTF-8    LC_PAPER=C.UTF-8       LC_NAME=C             
 [9] LC_ADDRESS=C           LC_TELEPHONE=C         LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C   

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.13.0 tibble_3.0.3     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5          rstudioapi_0.11     magrittr_1.5.0.9000 tidyselect_1.1.0    munsell_0.5.0      
 [6] colorspace_1.4-1    R6_2.4.1            rlang_0.4.7         fansi_0.4.1         plyr_1.8.6         
[11] dplyr_1.0.2         tools_4.0.3         grid_4.0.3          gtable_0.3.0        cli_2.0.2          
[16] ellipsis_0.3.1      assertthat_0.2.1    lifecycle_0.2.0     crayon_1.3.4        purrr_0.3.4        
[21] ggplot2_3.3.2       vctrs_0.3.4         glue_1.4.2          compiler_4.0.3      pillar_1.4.6       
[26] generics_0.0.2      scales_1.1.1        pkgconfig_2.0.3    

Any case, thank you for this great package.

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.