Max outgoing connection limit with pooling
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 6.9k
- Forks
- 1.3k
- Avg merge
- 13d 21h
- Merged PRs (30d)
- 1
Description
Problem:
* There is no way of limiting connections without killing them (resource manager will make them fail if above threshold)
* To applications above, this looks as if the remote peer is not dialable at all, but it is more of a local resource problem.
* In order to work with shitty routers and ISPs, we need to reduce the total amount of active connections, but without making peers undialable in the process. The solution is to place connections in a waiting pool. Applications can have more fine-tuned control on how long they want to wait for a dial to happen before giving up on it, rather than receiving an error immediately.
Solution:
* Add outgoing-connection pooling. Dials can happen normally while the total number of connections is below a threshold. When the threshold is crossed, dials will wait until existing connections are closed and leave space for new ones.
Example:
* For example, if a router only supports 1100 concurrent connections, and we set the threshold to 1000, and we need to perform a network crawl that requires a total of 7000 connections, we can ensure that we never have more than 1000 outgoing connections without breaking the network crawl (if we were to use resource manager limits) nor the router. This problem exists now with the accelerated DHT client.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the outgoing dial path, resource manager connection limits, and the accelerated DHT client mentioned in the issue. Determine where dials can wait for capacity and how callers should control their wait duration. Done means outgoing connections remain at or below the configured threshold while queued dials proceed when existing connections close.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100