Citus enforcing sequential mode even if there is a foreign key between Citus local tables
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
Hi, I've stumbled on a potential bug in interaction between citus-local (probably reference as well) and distributed tables.
### Setup
Try this setup (I used v11.2.0):
```sql
CREATE TABLE distributed (id SERIAL PRIMARY KEY);
CREATE TABLE local (id SERIAL PRIMARY KEY);
SELECT citus_add_local_table_to_metadata('local');
SELECT create_distributed_table('distributed', 'id');
CREATE TABLE local_dependent (id SERIAL PRIMARY KEY, fk INT REFERENCES local (id));
INSERT INTO distributed SELECT * FROM generate_series(1,10);
```
There is no relation between the `distributed` and any of the local tables, only between the local tables themselves.
### Problem
Now, if I modify the `distributed` table and the `local` table in the same transaction, I'll get an error.
```sql
BEGIN;
DELETE FROM distributed;
INSERT INTO local VALUES(DEFAULT);
-- ERROR: cannot modify table "local" because there was a parallel operation on a distributed table
-- DETAIL: When there is a foreign key to a reference table or to a local table, Citus needs to perform all operations over a single connection per node to ensure consistency.
-- HINT: Try re-running the transaction with "SET LOCAL citus.multi_shard_modify_mode TO 'sequential';"
```
I'd expect this to succeed because the database doesn't ensure any data integrity between the two tables I tried to modify.
### Other observations
It's interesting that if the table `local_dependent` doesn't exist at all, the transaction above succeeds.
Contributor guide
Assessment
This issue has not been assessed yet.