citusdata / citusdata/pg_shard
Support having multiple head nodes
- Dominant language
- C
- Stars
- 1.1k
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
The missing feature for pg_shard-as-OLTP-big-data tool is the ability to have multiple head nodes so that you can support thousands of connections. This shouldn't be too hard to solve, and may actually be solvable with existing PostgreSQL 9.4 tools.
1. replicate all of the metadata tables only: these don't get updated that often, so [BDR](https://wiki.postgresql.org/wiki/BDR_Project) ought to be adequate for it.
2. streaming replication option. This one is hampered by (a) needing to figure out which node is the master, and (b) pg_shards' use of temporary tables.
3. FDW option: have metadata be an FDW back to the master node. Likely to add latency and complexity. Also fails to provide us with easy failover from the master node.
Contributor guide
Research direction
This is a design proposal for multiple head nodes in pg_shard, with no files or tests identified. Start by comparing the three mentioned approaches—BDR metadata replication, streaming replication, and an FDW to the master—against master discovery, failover, and temporary-table use. Done would require a selected approach and defined behavior for those constraints.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100