parallel copy
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- r
- Domain
- performance
Research direction
Start by reviewing the timing experiment at commit fbb651005e9a1b51cccbcefc29f39d90fcd376f5 and the long-vector work in issue #3957. Determine where vector copying is implemented and how integer, logical, and double vectors are covered. Done means a supported parallel copy approach is implemented and its performance and variance are measured against the listed baselines.
Written by the indexing model from the issue text.
Description
This might be not that high priority now, but once long vectors will be supported #3957, then having faster copy might be saving more time.
I made an experiement how much we can save on 2e9 integer vector (7.5 GB in memory).
It should be the same for logical vector, double vector (8 bytes) should get even more speed up.
- plain for loop: 5.05s
- memcpy: 5.3s
- omp for static 40th one-by-one: 0.55s
- omp for dynamic 40th memcpy 40 batches: 0.52s
memcpy timings have significantly bigger variance
code used for timings: https://gitlab.com/jangorecki/data.table/-/commit/fbb651005e9a1b51cccbcefc29f39d90fcd376f5
- Dominant language
- R
- Stars
- 3.9k
- Forks
- 1.1k
- Avg merge
- 14h 4m
- Merged PRs (30d)
- 4
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Rdatatable/data.table
-
as.data.table() recurses without end on a survival::Surv object (or any data.frame carrying one) Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Rdatatable/data.table#7887 ·
-
consistency tests
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#7853 · 3 comments ·
-
internals
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#6938 · 1 comment ·
-
encoding fread
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Rdatatable/data.table#5179 · 8 comments ·
-
documentation programming
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Rdatatable/data.table#3199 · 3 comments ·
All issues in Rdatatable/data.table
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
r-lib/pkgdepends#485 · 3 comments ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
beginners blocker
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enviPathR OpenBuild Error Build OK Build Warning policies-accepted pre-review precheck-passed
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Bioconductor/BiocContributions#207 · 6 comments ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
datacarpentry/semester-biology#1255 ·