citusdata / citusdata/citus

Citus switches to sequential execution unnecessarily

Open
#4,138 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

```sql
CREATE TABLE reference_table1(a int unique, b int);
CREATE TABLE reference_table2(a int unique, b int);
SELECT create_reference_table('reference_table1');
SELECT create_reference_table('reference_table2');
ALTER TABLE reference_table1 ADD CONSTRAINT fkey_ref_to_ref FOREIGN KEY(a) REFERENCES reference_table2(a) ON DELETE RESTRICT;

set client_min_messages to DEBUG1;
BEGIN;
INSERT INTO reference_table2 VALUES (1);
DEBUG: switching to sequential query execution mode
DETAIL: Reference table "reference_table2" is modified, which might lead to data inconsistencies or distributed deadlocks via parallel accesses to hash distributed tables due to foreign keys. Any parallel modification to those hash distributed tables in the same transaction can only be executed in sequential query execution mode
```

This is the case even if no distributed tables are referencing to `reference_table2` (directly or transitively).
It's probably because [this](https://github.com/citusdata/citus/blob/366461ccdb7c1bbdfef0a8ab1c12632ee083f135/src/backend/distributed/transaction/relation_access_tracking.c#L749-L750) check doesn't consider if at least one of the referencing relations are distributed.

Even more, we might error out if we first access to an unrelated (not referenced/referencing) distributed table:
```sql
CREATE TABLE distributed_table(a int, b int);
SELECT create_distributed_table('distributed_table', 'a');

BEGIN;
SELECT * FROM distributed_table;
INSERT INTO reference_table2 values (1);
ERROR: cannot modify reference table "reference_table2" because there was a parallel operation on a distributed table
DETAIL: When there is a foreign key to a reference table, Citus needs to perform all operations over a single connection per node to ensure consistency.
HINT: Try re-running the transaction with "SET LOCAL citus.multi_shard_modify_mode TO 'sequential';"
```

Contributor guide

Open the contributing guide

Research direction

Start with the SQL reproductions in the issue and inspect src/backend/distributed/transaction/relation_access_tracking.c around lines 749-750. Trace how foreign-key references are classified when reference tables and distributed tables are accessed in one transaction. Done means unrelated reference-table modifications no longer force sequential execution or error after an unrelated distributed-table access, while relevant foreign-key cases retain their consistency behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql, sql
Domain
backend, databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.