citusdata / citusdata/citus

PG19: support distributed REPACK CONCURRENTLY

Open
#8,772 3 comments 0 reactions 0 assignees View on GitHub
pg19_features
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

Part of #8597 (Phase 3 — later feature integration). Depends on #8756.

## Current state

PG19 renames `ClusterStmt` to `RepackStmt`. #8624 / `6813788ed` dispatches `T_RepackStmt` and propagates plain `REPACK` to shards through the existing `CLUSTER` shard-DDL path. `REPACK CONCURRENTLY` and `REPACK ... ANALYZE` are **rejected with an error** (`src/backend/distributed/commands/cluster.c`, `RepackStmtOptionEnabled()` guard near line 111).

That behaviour is correct and fail-closed. #8756 covers refining *when* the rejection happens relative to lock acquisition.

This issue covers the remaining depth gap: actually supporting `REPACK CONCURRENTLY` on distributed tables.

## Why it is not a simple propagation

Unlike plain `REPACK`, `CONCURRENTLY` is not a self-contained DDL statement that can be forwarded per shard and forgotten:

- It runs work in a **background worker**, not in the issuing backend.
- It creates a **temporary logical replication slot** named `pg_repack_` to capture concurrent changes.
- It therefore requires `wal_level = logical` on every node that runs it.
- Slot count is bounded by a new **`PGC_POSTMASTER`** GUC, `max_repack_replication_slots` (`src/include/replication/slot.h:327`). Postmaster-level means it cannot be raised without a restart.

## What distributed support has to decide

1. **Slot budgeting.** A distributed `REPACK CONCURRENTLY` fans out to every shard placement on a node. Concurrent repacks on one worker can exhaust `max_repack_replication_slots`, and the limit cannot be raised at runtime. Fan-out must be throttled, or the operation must be serialized per node.
2. **`wal_level` precondition.** Citus must verify `wal_level = logical` on all nodes and fail with a clear message rather than surfacing a per-shard slot error.
3. **Partial failure.** Background-worker execution means shard operations complete asynchronously and independently. Some shards can succeed while others fail, leaving the table repacked unevenly. Reporting and cleanup semantics need defining.
4. **Interaction with Citus's own maintenance** — rebalancer moves, shard splits, and logical replication already in flight.

## Priority

Low. Nothing is silently wrong today; users get an explicit error. This is a capability gap, not a correctness gap, and it should not gate the `pg19-support` → `main` merge.

Contributor guide

Open the contributing guide

Research direction

Start with src/backend/distributed/commands/cluster.c, especially the RepackStmtOptionEnabled() guard, and read the related slot definition at src/include/replication/slot.h:327. Review the dependencies and the listed slot, wal_level, partial-failure, and maintenance concerns; done requires a defined and implemented distributed REPACK CONCURRENTLY design with clear failure and cleanup behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.