citusdata / citusdata/citus

create_distributed_table_concurrently is slow compared to create_distributed_table with many partitions

Open
#6,380 0 comments 0 reactions 0 assignees View on GitHub
dev_reported enhancement
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

I don't think there is anything broken or such. Mostly opened as enhancement task. All tests done on my local machine. There is no activity in the cluster during the tests. This includes the improvements in #6314

When I use `create_distributed_table_concurrently()`:

```SQL

CREATE TABLE time_series_events (event_time timestamp, event int, user_id int)
PARTITION BY RANGE (event_time);

-- create 8 months daily partitions
SELECT create_time_partitions(table_name:= 'time_series_events',
partition_interval:= '1 day',
end_at:= '2022-06-10',
start_from:= '2021-10-10');

SELECT create_distributed_table_concurrently('time_series_events', 'user_id');
Time: 157733.348 ms (02:37.733)
```

When I use `create_distributed_table()`

```SQL
CREATE TABLE time_series_events (event_time timestamp, event int, user_id int)
PARTITION BY RANGE (event_time);

-- create 8 months daily partitions
SELECT create_time_partitions(table_name:= 'time_series_events',
partition_interval:= '1 day',
end_at:= '2022-06-10',
start_from:= '2021-10-10');

SELECT create_distributed_table('time_series_events', 'user_id');
Time: 25457.045 ms (00:25.457)
```

So, there is ~9x difference.

On top of that, while the partitioning hierarchy is created (e.g., ALTER TABLE .. ATTACH PARTITION happens on the shards), the modifications are blocked as we already do `BlockWritesToShards()` at that point. This might cause non-negligible downtime.

Contributor guide

Open the contributing guide

Research direction

Start by running the two SQL reproductions for create_distributed_table_concurrently() and create_distributed_table() with the daily partitions described. Trace those entry points through partition hierarchy creation, including ALTER TABLE ... ATTACH PARTITION and BlockWritesToShards(), while considering the improvements in #6314. Done means identifying the cause of the slowdown and reducing runtime and write-blocking impact without breaking the concurrent operation.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, sql
Domain
backend, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.