fdbcli kill failure with TLS possibly due to slow task
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
We've seen some instances of using `fdbcli` to kill processes in the cluster failing due to `fdbcli` being unable to establish connections to processes in the cluster. More specifically, it succeeded in killing only those processes that it was already connected to, which were the coordinators and the cluster controller.
There is another issue (#2209) that would have `fdbcli` connect to all processes prior to issuing the kill request, but unless we deal with the slow task it may run into similar troubles.
Specifically, the problem seems to be that when using TLS, the `ASIOReactor` spends a sizable amount of time in the react function per connection. When there are a lot of connections at once, this can take long enough that the processes we are connecting to time out the connection without having successfully established it completely.
Contributor guide
Research direction
Start with the fdbcli process-kill flow and the ASIOReactor react function mentioned in the report. Investigate TLS connection establishment when many processes are contacted at once, and verify the kill operation can connect to and terminate all intended processes without connection timeouts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, distributed-systems, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100