Process creation on Windows is needlessly serialised
- Dominant language
- Haskell
- Stars
- 91
- Forks
- 91
- PR merge metrics
- No merged PRs in 30d
Description
When `close_fds = True`, we can explicitly pass the processes that the child is allowed to inherit. This completely avoids the problem of [GHC bug #2650](https://gitlab.haskell.org/ghc/ghc/-/work_items/2650) that made locking necessary.
In a microbenchmark, spawning 200 processes with `close_fds = True` concurrently takes about 1.5s on my machine, with over 40% of the time (0.7s) wasted due to waiting on the lock.
While working on speeding up GHC's downsweep pass, I found that on Windows a lot of time was spent on process spawning (e.g. to run the C preprocessor), and that this serialisation was a significant component of that. In a rebuild of the `vulkan` package, the downsweep pass spawns a huge amount of subprocesses, and removing this locking takes the time spent in downsweep from 31s to 27s (4s total timesave).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the Windows process-spawning path for close_fds=True and review the explicit child-inheritance approach described in the issue, alongside GHC bug #2650. Validate the change with the concurrent process-spawning microbenchmark described here; done means the unnecessary lock serialization is removed without changing which handles the child inherits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- haskell
- Domain
- operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100