haskell / haskell/process

Process creation on Windows is needlessly serialised

Open
#381 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
91
Forks
91
PR merge metrics
No merged PRs in 30d

Description

When `close_fds = True`, we can explicitly pass the processes that the child is allowed to inherit. This completely avoids the problem of [GHC bug #2650](https://gitlab.haskell.org/ghc/ghc/-/work_items/2650) that made locking necessary.

In a microbenchmark, spawning 200 processes with `close_fds = True` concurrently takes about 1.5s on my machine, with over 40% of the time (0.7s) wasted due to waiting on the lock.

While working on speeding up GHC's downsweep pass, I found that on Windows a lot of time was spent on process spawning (e.g. to run the C preprocessor), and that this serialisation was a significant component of that. In a rebuild of the `vulkan` package, the downsweep pass spawns a huge amount of subprocesses, and removing this locking takes the time spent in downsweep from 31s to 27s (4s total timesave).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the Windows process-spawning path for close_fds=True and review the explicit child-inheritance approach described in the issue, alongside GHC bug #2650. Validate the change with the concurrent process-spawning microbenchmark described here; done means the unnecessary lock serialization is removed without changing which handles the child inherits.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.