ruby / ruby/net-http

Feature Request: Add support for persistent keep-alive connections with auto healing (that can be used to create connection pools)

Open
#222 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
148
Forks
95
Avg merge
10h 54m
Merged PRs (30d)
4

Description

Hello,

We have a scenario (push service) where we need to send thousands of HTTP requests per second to the same host. We use Sidekiq to scale this workload. The problem is that every request (in a Sidekiq Job) starts a new connection, makes the request, closes the connection.

It would be a great performance improvement if we could use a gem like this https://github.com/mperham/connection_pool to create a connection pool for HTTP requests.

  1. Define a connection pool
  2. Take a connection from the connection pool and use it from a Sidekiq Job to make the request
  3. Put back the connection in the connection pool
  4. The connection can be reused by other Sidekiq Jobs to make other HTTP requests to the same host.

The problem, as described in the ConnectionPool readme, is this:

connections should be self-repairing.

For example if a keep alive connection is closed on the server side, this library gives errors, but does not retry / repair the connection automatically.

It would be great to have support for persistent keep-alive connections and auto healing directly in Ruby.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading Net::HTTP's existing connection lifecycle and the connection_pool README referenced in the issue. Define how persistent connections are pooled, reused across Sidekiq jobs, and repaired after server-side closure; completion would require agreed behavior and tests for these cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.