futureverse / futureverse/future

Note on mixing threading and forking

Open
#405 2 comments 1 reaction 0 assignees View on GitHub
knowledge-base
Dominant language
R
Stars
1k
Forks
92
PR merge metrics
No merged PRs in 30d

Description

Just wanted to leave a comment [somewhere](https://xkcd.com/979/) regarding an issue I was having using `plan(multicore)` on Linux. Specifically, the code would unpredictably deadlock.

I **think** I've figured out the problem - I had MKL with threading set up as the back-end for linear algebra. And threading + fork is apparently [known](https://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them) to be [bad](https://blog.revolutionanalytics.com/2015/10/edge-cases-in-using-the-intel-mkl-and-parallel-programming.html).

So if you're using `plan(multicore)` on Linux and it's sporadically hanging, check that you're not accidentally using threads.

Contributor guide

Open the contributing guide

Research direction

Start by locating the documentation or guidance for plan(multicore) on Linux; no specific file or test is identified in the issue. Record the warning that threaded linear algebra backends such as MKL can cause forked workers to deadlock, with the linked references as context. Done means users can discover this troubleshooting note when using plan(multicore).

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
hpc, operating-systems
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.