Connection management improvements for multi-database setup
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
This issue is more of an invitation to a technical discussion than a solid proposal.
Currently, the main issue with having multiple `DATABASE`s in a cluster is connection management which is not adapted to this. The issue can be split into two parts:
1. The most problematic areas are the transaction recovery and distributed deadlock detection mechanisms that do not respect `citus.max_shared_pool_size` and open at least `n * d` connections per worker, where n -- number of nodes in cluster, d -- number of `DATABASE`s
2. Even if they respect the `citus.max_shared_pool_size`, this limit is applied per `DATABASE`, meaning that it will still require `n * d` connections.
In order to overcome this, I propose to:
1. Make `citus.max_shared_pool_size` cluster-wide. Per-database setting then may be used to set a connection quota for a database within the global `citus.max_shared_pool_size`
2. Make the transaction recovery and distributed deadlock detection respect the improved `citus.max_shared_pool_size`.
Since those changes make sense mostly for the multi-database setup, they may be enabled by a single GUC, like `citus.multi_database`, or have a separate GUC for the behaviour of `citus.max_shared_pool_size`, transaction recovery, and distributed deadlock detection.
Contributor guide
Assessment
This issue has not been assessed yet.