citusdata / citusdata/citus

Use TRUNCATE + COPY FREEZE when copying a shard

Open
#7,403 2 comments 0 reactions 1 assignee Claimed by @m3hm3t View on GitHub
good first issue performance
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

In `src/backend/distributed/operations/worker_shard_copy.c` file contains our main COPY logic for for shard moves and shard splits. After talking with @DimCitus I realized that we could use the [`FREEZE` option of `COPY`](https://www.postgresql.org/docs/current/sql-copy.html) to reduce the the need for heavy vacuuming after the copy is done. To benefit from the freeze option it's required to truncate the target table in the same transaction as the copy. This is fine for all our (current) use cases, because the target shard has just been created and is thus empty.

What I think is needed to achieve this:
1. Add the `FREEZE` option to the COPY command that we generate in `ConstructShardCopyStatement`
2. Add the `FREEZE` option to the list of options created in `LocalCopyToShard`
3. Start a transaction in `ConnectToRemoteAndStartCopy`
4. Truncate the table in `ConnectToRemoteAndStartCopy` (before starting the COPY)
5. End the transaction in `ShardCopyDestReceiverShutdown`

Apart from the actual implementation this needs tests to see that indeed vacuum is not necessary on the new table after a shard move.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.