citusdata / citusdata/citus_docs

Documentation request: Citus cluster migration using logical replication (publication + subscription)

Open
#1,125 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
CSS
Stars
58
Forks
60
PR merge metrics
No merged PRs in 30d

Description

## Why are we implementing it? (sales eng)

The documentation is missing currently. By properly documenting migration methods in different scenarios helps users trust using Citus as a large-scale data solution.

### What are the typical use cases?

Migrate a self-hosted Citus cluster into new setup (perhaps more powerful hardware, different geographical location or using new networking architecture - hopefully with minimal downtime.

## How does this work? (devs)
On idea level
1. deploy a new Citus cluster having same number of workers as the original cluster 1..N
2. copy the data over and keep it in sync while clients keep on using the old cluster
3. switch over the clients to the new cluster

I presume step 2 could be achieved using publications & subscriptions of non-citus-metadata tables ("workload" tables) between each old and new worker pair N. Clusters will eventually become in-sync regarding workload data, but the new cluster would have its unique metadata (mainly `pg_dist_node`).

Using pubsub would allow installing the new cluster with newer PG version, having the PG version upgrade would be a "free" byproduct.

We've done very successfully a few of this kind of pubsub migrations into a newer PG versions with regular Patroni clusters, with downtime of single digit seconds, so it would be nice if same could be done with Citus as well.

### Corner cases, gotchas

### Are there relevant blog posts or outside documentation about the concept/feature?
I created an [issue](https://github.com/citusdata/citus/issues/7672) in the main Citus repo before knowing there exists this documentation repo as well.

More details about my specific use case there, and in a [Citus Slack message](https://citus-public.slack.com/archives/C0XRHT1KJ/p1721915374554979) I wrote earlier.

Contributor guide

Open the contributing guide

Research direction

Start by reading the migration details in Citus issue 7672 and the linked Slack message, then compare them with the existing Citus documentation. Document the publication-and-subscription migration scenario, including the stated cluster setup, synchronization, switchover, and corner cases; done means users can follow the procedure and understand its limitations.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.