citusdata / citusdata/citus

Aggressive locking on partitioned tables with multi shard modification queries

Open
#2,093 0 comments 0 reactions 0 assignees View on GitHub
1-2 days bug
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

In general, for various reasons, Citus prevents running multiple multi-shard modification queries on the same table concurrently (to be more precise, on the same shards).

While enabling partitioning on distributed tables, we followed the same rule. In addition to that, we prevented running a multi-shard modification query on the partitioned tables while a multi-shard modification is running on one of its partitions. The rationale was simple: We cannot know which partitions the query on the partitioned table would hit. Thus, prevent multiple concurrent modifications on the parent and its partitions.

Up until now, I think our rationale was OK. However, we realized that our implementation prevents one more thing: Running concurrent multi-shard modifications on different partitions. We should relax the locking in these situations because there seems no need to prevent it. Specifically, do it in `LockParentShardResourceIfPartition()`.

We should come up with a locking scheme such that concurrent modifications on the parent and one of its partitions are prevented but not two concurrent multi-shard modifications on the multiple partitions are prevented.

Steps to reproduce:

```SQL
CREATE TABLE partitioning_test(id int, time date) PARTITION BY RANGE (time);

-- create its partitions
CREATE TABLE partitioning_test_2009 PARTITION OF partitioning_test FOR VALUES FROM ('2009-01-01') TO ('2010-01-01');
CREATE TABLE partitioning_test_2010 PARTITION OF partitioning_test FOR VALUES FROM ('2010-01-01') TO ('2011-01-01');

SELECT create_distributed_table('partitioning_test', 'id');

-- session 1
BEGIN;
INSERT INTO partitioning_test_2009 SELECT * FROM partitioning_test_2009;

-- session 2
BEGIN;
-- ops, this is blocked unnecessarily
INSERT INTO partitioning_test_2010 SELECT * FROM partitioning_test_2010;

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.