Aggressive locking on partitioned tables with multi shard modification queries
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
In general, for various reasons, Citus prevents running multiple multi-shard modification queries on the same table concurrently (to be more precise, on the same shards).
While enabling partitioning on distributed tables, we followed the same rule. In addition to that, we prevented running a multi-shard modification query on the partitioned tables while a multi-shard modification is running on one of its partitions. The rationale was simple: We cannot know which partitions the query on the partitioned table would hit. Thus, prevent multiple concurrent modifications on the parent and its partitions.
Up until now, I think our rationale was OK. However, we realized that our implementation prevents one more thing: Running concurrent multi-shard modifications on different partitions. We should relax the locking in these situations because there seems no need to prevent it. Specifically, do it in `LockParentShardResourceIfPartition()`.
We should come up with a locking scheme such that concurrent modifications on the parent and one of its partitions are prevented but not two concurrent multi-shard modifications on the multiple partitions are prevented.
Steps to reproduce:
```SQL
CREATE TABLE partitioning_test(id int, time date) PARTITION BY RANGE (time);
-- create its partitions
CREATE TABLE partitioning_test_2009 PARTITION OF partitioning_test FOR VALUES FROM ('2009-01-01') TO ('2010-01-01');
CREATE TABLE partitioning_test_2010 PARTITION OF partitioning_test FOR VALUES FROM ('2010-01-01') TO ('2011-01-01');
SELECT create_distributed_table('partitioning_test', 'id');
-- session 1
BEGIN;
INSERT INTO partitioning_test_2009 SELECT * FROM partitioning_test_2009;
-- session 2
BEGIN;
-- ops, this is blocked unnecessarily
INSERT INTO partitioning_test_2010 SELECT * FROM partitioning_test_2010;
```
Contributor guide
Assessment
This issue has not been assessed yet.