tarantool / tarantool/tarantool
Get rid of cluster full mesh
Nobody has claimed this yet.
- Dominant language
- Lua
- Stars
- 3.7k
- Forks
- 419
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 88
Description
Get rid of whole cluster full mesh to avoid too big number of sockets. When each instance sees each other even in different replicasets, it will create too many threads and sockets according to current implementation. Now each connection with a master/slave means +1 relay/applier thread, +1 socket. So for a cluster with 100 instances there are 9900 threads and sockets, 99 per each instance. A possible solution is to make the links “weaker”. For example, interconnect instances of different replicasets either via UDP (hi, SWIM), or via temporary TCP connections, which are closed after too long idle time. Relay/applier threads should be removed and possibly merged into WAL thread, or iproto. This is a preparation for sharding integration into the core.
Follows #3983.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by mapping the current full-mesh connection implementation and its relay/applier threads, then review the preparation work in #3983. Done would require an agreed replacement for cross-replicaset links and a validated reduction in connections and threads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- databases, distributed-systems
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100