packrat symlink behavior causes errors when running many R sessions simultaneously
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 409
- Forks
- 89
- PR merge metrics
- No merged PRs in 30d
Description
Sorry, this is my last issue! (and thanks for all the help, packrat is a fantastic tool :)
I'm running simulations in R on a computer cluster, by running the same Rscript job on many nodes in parallel. However, since packrat deletes and recreates symlinks, it causes several errors across jobs (in a sporadic fashion).
I can work around this by staggering each job, so they don't begin at the same time. However, it seems like an option to disable the deletion and recreation of symlinks would bring peace of mind when turning packrat mode on! Going further, an option to disable the creation or deletion of anything, as if running packrat in "read-only" mode seems like it would be valuable for making packrat mode very simple and intuitive.
errors with base package symlinks
Error in if (nt == 1 && file.exists(to) && file.info(to)$isdir) to <- file.path(to, :
missing value where TRUE/FALSE needed
Calls: source ... setPackratModeOn -> afterPackratModeOn -> symlinkSystemPackages
In addition: Warning messages:
1: In readLines(file) :
cannot open compressed file '/jukebox/hasson/michael/dissertation/paper-pattern-similarity/packrat/lib-R/base/DESCRIPTION', probable reason 'No such file or
directory'
2: In value[[3L]](cond) :
Unable to read DESCRIPTION file associated with 'base' package
Execution halted
warnings with external package symlinks (I think I've gotten errors here also, but would need to start a bunch of jobs and cross my fingers)
Warning messages:
1: In file.symlink(from, to) :
cannot symlink '/usr/people/machow/R/library/RCurl' to '/jukebox/hasson/michael/dissertation/paper-pattern-similarity/packrat/lib-ext/RCurl', reason 'File ex
ists'
2: In file.symlink(from, to) :
cannot symlink '/usr/people/machow/R/library/bitops' to '/jukebox/hasson/michael/dissertation/paper-pattern-similarity/packrat/lib-ext/bitops/bitops', reason
'File exists'
3: In symlinkExternalPackages(project = project) :
The following external packages could not be linked into the packrat private library:
- 'RCurl', 'bitops'
4: In file.symlink(from, to) :
cannot symlink '/usr/people/machow/R/library/RCurl' to '/jukebox/hasson/michael/dissertation/paper-pattern-similarity/packrat/lib-ext/RCurl/RCurl', reason 'F
ile exists'
5: In file.symlink(from, to) :
cannot symlink '/usr/people/machow/R/library/bitops' to '/jukebox/hasson/michael/dissertation/paper-pattern-similarity/packrat/lib-ext/bitops/bitops', reason
'File exists'
6: In symlinkExternalPackages(project = project) :
The following external packages could not be linked into the packrat private library:
- 'RCurl', 'bitops'
I dug around a bit in the source code to figure out what was going on, and am happy to take a pass at implementing something if pointed in the right direction.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the setPackratModeOn -> afterPackratModeOn -> symlinkSystemPackages path shown in the error, then inspect symlinkExternalPackages for the external-package warnings. Done means concurrent R sessions no longer fail or warn because symlink operations race, with the requested read-only or symlink-preserving behavior defined and covered by verification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100