Weird behavior of setcolorder

Open
#4,690 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Unzip df.zip to obtain df.rds and run the annotated script in the issue to reproduce the mismatch after data.table::setcolorder() is called on a data.frame. Start by inspecting the setcolorder entry point and its handling of data.frames, then compare the original and column-selected objects. Done means the reordered values and column names agree for this reproduction.

Written by the indexing model from the issue text.

Description

by-reference

I have this weird behavior on a data.frame that is the result of RSQLite. I have attached it (df.zip) since this is the only way to reproduce the problem.

Please unzip before using it: github requires zipping the file.

Here is an annotated script that reproduces the problem with this file:

# load the data.frame
df <- readRDS('df.rds')
class(df)
# [1] "data.frame"

# there are many columns, but the first few are enough to show the problem
df[, 1:5]
#     estate_id mod_timestamp cust_id cust_isOwner cust_publicInfo.name
# 1 -2147474620    1596557895 1575296            1                 <NA>

# now I want to sort the columns (by reference?)
data.table::setcolorder(df, neworder = sort(colnames(df)))

# now let's look at the first few columns
df[, 1:5]
#   estate_id mod_timestamp cust_id cust_isOwner cust_publicInfo.name
# 1   1575296             1       0         <NA>                 <NA>

# (!!) you see here that the fields have changed (i.e. sorted), but not the column names (!!)

# let us take a copy of this table by (!) selecting the columns this time (!)
df2 <- readRDS('df.rds')[, colnames(readRDS('df.rds'))]

# now do the same: order columns
data.table::setcolorder(df2, neworder = sort(colnames(df2)))
df2[, 1:5]
#   cust_id cust_isOwner cust_publicInfo.hasIpiNo cust_publicInfo.ipiNo cust_publicInfo.name
# 1 1575296            1                        0                  <NA>                 <NA>

# (!!) now the columns and fields are ordered as they should (!!)

Any clue on what is going on here? Both readRDS('df.rds') and readRDS('df.rds')[, colnames(readRDS('df.rds'))] are identical, so I don't understand what is causing this column ordering discrepancy.

sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=Dutch_Belgium.1252  LC_CTYPE=Dutch_Belgium.1252    LC_MONETARY=Dutch_Belgium.1252 LC_NUMERIC=C                   LC_TIME=Dutch_Belgium.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_3.6.3    tools_3.6.3       data.table_1.13.0

Note: converting df to data.table solve the problem, but begs the question. Why doesn't setcolorder() work with this specific data.frame.

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.