Feature Request: Add support for persistent keep-alive connections with auto healing (that can be used to create connection pools)
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 148
- Forks
- 95
- Avg merge
- 10h 54m
- Merged PRs (30d)
- 4
Description
Hello,
We have a scenario (push service) where we need to send thousands of HTTP requests per second to the same host. We use Sidekiq to scale this workload. The problem is that every request (in a Sidekiq Job) starts a new connection, makes the request, closes the connection.
It would be a great performance improvement if we could use a gem like this https://github.com/mperham/connection_pool to create a connection pool for HTTP requests.
- Define a connection pool
- Take a connection from the connection pool and use it from a Sidekiq Job to make the request
- Put back the connection in the connection pool
- The connection can be reused by other Sidekiq Jobs to make other HTTP requests to the same host.
The problem, as described in the ConnectionPool readme, is this:
connections should be self-repairing.
For example if a keep alive connection is closed on the server side, this library gives errors, but does not retry / repair the connection automatically.
It would be great to have support for persistent keep-alive connections and auto healing directly in Ruby.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading Net::HTTP's existing connection lifecycle and the connection_pool README referenced in the issue. Define how persistent connections are pooled, reused across Sidekiq jobs, and repaired after server-side closure; completion would require agreed behavior and tests for these cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100