Need a way to control copy behavior when using :=

Open
#2,357 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Start with the data[, names(cols) := cols] entry point and compare it with the c(as.list(data), cols); setDT(data) workaround. Determine how := currently handles copying and define tests for explicit user-controlled behavior; done means adding the columns without unnecessary copying while preserving their values.

Written by the indexing model from the issue text.

Description

performance

I'm working with data, a data.table of 50M rows and 60 columns. I'll compute more than 100 new columns to add to the data.table.

cols <- mclapply(1:100, function(i) nrow(data), mc.cores = 40)
names(cols) <- paste0("x", seq_along(cols))

system.time(data[, names(cols) := cols])

I'll take more than 30 seconds. However, cols are no longer used so it'll be safe not to copy it when performing :=. I wonder if user should be allowed to control the copy behavior of := which is by default TRUE in this case.

A simple walk-around is

data <- c(as.list(data), cols)
setDT(data)

This only takes 0.3 seconds without copying any columns.

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.