Use the qs package for file transfers
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 184
- Forks
- 53
- Avg merge
- 7d 2h
- Merged PRs (30d)
- 1
Description
Ref: https://github.com/HenrikBengtsson/future.batchtools/issues/36
I and others are finding that the job-submission rate when using batchtools via future.batchtools can be quite slow.
Is it feasible to speed up the transmission of the files using the qs package? https://cran.r-project.org/web/packages/qs/index.html
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked future.batchtools issue and the qs package documentation to understand the reported transfer bottleneck and proposed serialization approach. Trace batchtools’ current file-transfer path, then define completion as a measured determination of whether qs improves submission speed, with any required implementation and regression coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- hpc, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100