citusdata / citusdata/citus

Documentation request: Citus cluster migration

Open
#7,672 0 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

TLDR - I'd like to have documentation covering Citus cluster migration scenarios

Our background (the non-TLDR part) is covered in [Citus Slack](https://citus-public.slack.com/archives/C0XRHT1KJ/p1721915374554979)

To successfully migrate a complete Citus cluster the needed steps are somewhat clear:
- deploy a new cluster
- copy the data over and keep it in sync
- switch over the clients to the new cluster

I believe it's possible to replicate a (current) Citus cluster into empty new Citus cluster (with same number of workers) using PUBLICATION + SUBSCRIPTION across each old and new workers, and across the old and new coordinators. By reading the [documentation about CDC](https://docs.citusdata.com/en/v12.1/develop/integrations.html) the idea is clear - just create pubs+subs not for ALL TABLES, but for the non-metadata user tables. But as we're replicating each worker and coordinators separately it's unclear if [citus.enable_change_data_capture](https://docs.citusdata.com/en/v12.1/develop/api_guc.html#enable-change-data-capture) brings anything on the table (as we want to each target worker / shard to get 1:1 changes from the source)

It would be awesome if this kind of "migrate your Citus into new hardware" type of scenario would be covered in the documentation.

Contributor guide

Open the contributing guide

Research direction

Start with the linked CDC documentation and the sections on PUBLICATION, SUBSCRIPTION, and citus.enable_change_data_capture. Determine the supported steps for deploying a new cluster, copying and synchronizing data, and switching clients, including how coordinators and workers are handled. Done means the documentation clearly covers the migration scenario and explains whether the setting is needed.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql
Domain
databases, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.