capistrano / capistrano/sshkit
Feature: Configuration to set maximum parallelization of :parallel runner
- Dominant language
- Ruby
- Stars
- 1.2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Description
There have been multiple requests to set an upper limit on the number of `git` operations that are executed in parallel in the default Capistrano git strategy. Similarly, users are also asking for a limit on the number of parallel `bundle install` executions in `capistrano-bundler`.
What these tasks have in common is that they all use the default `:parallel` runner provided by SSHKit. When using Capistrano to deploy to a large number of servers, firing off these operations to all servers in parallel can overload shared resources like a git server or private gem repository.
Rather than implement rate limiting for each SCM, capistrano-bundler, etc., I feel like a more general solution should be provided by SSHKit itself.
My proposal would be to change the implementation of the `:parallel` runner to essentially be a subclass of the `:groups` runner, except with defaults of `wait: 0` and `limit: INFINITY`. Then, if a user wants to limit the amount of parallelization, they could simply do this:
```ruby
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { limit: 10 }
```
If sharing implementation and configuration keys between `:parallel` and `:groups` is too confusing, then perhaps the `:parallel` runner could use a different configuration key (but to the same effect):
```ruby
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { threads: 10 }
```
Thoughts?
See also:
* https://github.com/capistrano/capistrano/pull/1957
* https://github.com/capistrano/capistrano/issues/1058
* https://github.com/capistrano/bundler/issues/94
* https://github.com/capistrano/bundler/pull/93
Contributor guide
Assessment
This issue has not been assessed yet.