Chunks
- Dominant language
- JavaScript
- Stars
- 2.4k
- Forks
- 97
- PR merge metrics
- No merged PRs in 30d
Description
There are generally a couple different ways to run tools in a Bolt repo:
- Globally (across all workspaces at once)
- By Workspace
- Parallel
- Chunks by dependency tree
- Serial
When you have a tool that could either run globally or by workspace, you have to
make a tradeoff decision:
- Does the tool take a long startup time? By workspace is probably slower, having to startup over and over.
- Does the tool have a long process time? Globally is probably slower, would benefit from parallelization.
Right now we don't offer anything in between these two extremes. The different
"by workspace" modes don't help at all because they all spawn a new process for
every workspace.
```
(./) $ tool "./packages/*"
```
```
(./packages/a) $ tool "."
(./packages/b) $ tool "."
(./packages/c) $ tool "."
(./packages/d) $ tool "."
```
But what if we could break workspaces into "chunks" and spawn a smaller number
of processes?
```
(./) $ tool "./{packages/a,packages/b}"
(./) $ tool "./{packages/c,packages/d}"
```
This could allow us to increase parallelism of a tool while also reducing the
number of processes that it starts up.
## CLI
I would focus on just when the `--parallel` flag is on initially.
```
bolt workspaces exec --parallel --chunks -- tool "()"
```
If you wanted to limit the number of items to place in a chunk at a time:
```
bolt workspaces exec --parallel --chunks --max-chunk-size 10 -- tool "()"
```
Contributor guide
Assessment
This issue has not been assessed yet.