capistrano / capistrano/sshkit

Feature: Configuration to set maximum parallelization of :parallel runner

Open
#414 10 comments 0 reactions 0 assignees View on GitHub
discuss! new feature
Dominant language
Ruby
Stars
1.2k
Forks
257
PR merge metrics
No merged PRs in 30d

Description

There have been multiple requests to set an upper limit on the number of `git` operations that are executed in parallel in the default Capistrano git strategy. Similarly, users are also asking for a limit on the number of parallel `bundle install` executions in `capistrano-bundler`.

What these tasks have in common is that they all use the default `:parallel` runner provided by SSHKit. When using Capistrano to deploy to a large number of servers, firing off these operations to all servers in parallel can overload shared resources like a git server or private gem repository.

Rather than implement rate limiting for each SCM, capistrano-bundler, etc., I feel like a more general solution should be provided by SSHKit itself.

My proposal would be to change the implementation of the `:parallel` runner to essentially be a subclass of the `:groups` runner, except with defaults of `wait: 0` and `limit: INFINITY`. Then, if a user wants to limit the amount of parallelization, they could simply do this:

```ruby
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { limit: 10 }
```

If sharing implementation and configuration keys between `:parallel` and `:groups` is too confusing, then perhaps the `:parallel` runner could use a different configuration key (but to the same effect):

```ruby
# Limit the default :parallel runner to 10 threads
SSHKit.config.default_runner_config = { threads: 10 }
```

Thoughts?

See also:

* https://github.com/capistrano/capistrano/pull/1957
* https://github.com/capistrano/capistrano/issues/1058
* https://github.com/capistrano/bundler/issues/94
* https://github.com/capistrano/bundler/pull/93

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.