futureverse / futureverse/future
Note on mixing threading and forking
- Dominant language
- R
- Stars
- 1k
- Forks
- 92
- PR merge metrics
- No merged PRs in 30d
Description
Just wanted to leave a comment [somewhere](https://xkcd.com/979/) regarding an issue I was having using `plan(multicore)` on Linux. Specifically, the code would unpredictably deadlock.
I **think** I've figured out the problem - I had MKL with threading set up as the back-end for linear algebra. And threading + fork is apparently [known](https://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them) to be [bad](https://blog.revolutionanalytics.com/2015/10/edge-cases-in-using-the-intel-mkl-and-parallel-programming.html).
So if you're using `plan(multicore)` on Linux and it's sporadically hanging, check that you're not accidentally using threads.
Contributor guide
Research direction
Start by locating the documentation or guidance for plan(multicore) on Linux; no specific file or test is identified in the issue. Record the warning that threaded linear algebra backends such as MKL can cause forked workers to deadlock, with the linked references as context. Done means users can discover this troubleshooting note when using plan(multicore).
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- hpc, operating-systems
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100