When using rules on citus tables, assertion crash or error is possible
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
```sql
create schema test;
set search_path to test;
create table rtest_emp (ename char(20), salary money);
create table rtest_emplog (ename char(20), who name, action char(10), newsal money, oldsal money);
select create_reference_table('rtest_emp');
select create_reference_table('rtest_emplog');
create rule rtest_emp_ins as on insert to rtest_emp do
insert into rtest_emplog values (new.ename, current_user,
'hired', new.salary, '0.00');
```
```sql
insert into rtest_emp values ('wiecc', '5000.00');
```
Considering above tables and insert rule, we break `insert` execution in three different cases:
1) If we are using citus in single node -only the coordinator is added with `groupId := 0`, below insert crashes due to assertion failure in local execution (`ShouldExecuteTasksLocally`) with the following backtrace:
```c
#2 0x0000000000aae380 in ExceptionalCondition (conditionName=0x7fa7f33b7968 "!(isValidLocalExecutionPath)", errorType=0x7fa7f33b7762 "FailedAssertion",
fileName=0x7fa7f33b7723 "executor/local_executor.c", lineNumber=677) at assert.c:54
#3 0x00007fa7f331edf6 in ShouldExecuteTasksLocally (taskList=0x18b93b0) at executor/local_executor.c:677
#4 0x00007fa7f331356a in CreateDistributedExecution (modLevel=ROW_MODIFY_COMMUTATIVE, taskList=0x18b93b0, paramListInfo=0x0, targetPoolSize=16,
defaultTupleDest=0x18b96d8, xactProperties=0x7ffcf996fff0, jobIdList=0x0) at executor/adaptive_executor.c:1126
#5 0x00007fa7f3312ca9 in AdaptiveExecutor (scanState=0x1894e70) at executor/adaptive_executor.c:732
#6 0x00007fa7f33180e8 in CitusExecScan (node=0x1894e70) at executor/citus_custom_scan.c:224
#7 0x0000000000732cc9 in ExecCustomScan (pstate=0x1894e70) at nodeCustom.c:116
#8 0x000000000071a5d6 in ExecProcNodeFirst (node=0x1894e70) at execProcnode.c:445
#9 0x000000000070f414 in ExecProcNode (node=0x1894e70) at ../../../src/include/executor/executor.h:239
#10 0x0000000000711c4f in ExecutePlan (estate=0x1894ba8, planstate=0x1894e70, use_parallel_mode=false, operation=CMD_INSERT, sendTuples=false,
numberTuples=0, direction=ForwardScanDirection, dest=0x18ab808, execute_once=true) at execMain.c:1646
#11 0x000000000070fa51 in standard_ExecutorRun (queryDesc=0x189dce8, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:364
#12 0x00007fa7f332087c in CitusExecutorRun (queryDesc=0x189dce8, direction=ForwardScanDirection, count=0, execute_once=true)
at executor/multi_executor.c:202
#13 0x000000000070f874 in ExecutorRun (queryDesc=0x189dce8, direction=ForwardScanDirection, count=0, execute_once=true) at execMain.c:306
#14 0x0000000000923abc in ProcessQuery (plan=0x18ab4a0, sourceText=0x1702ad8 "insert into rtest_emp values ('wiecc', '5000.00');", params=0x0,
queryEnv=0x0, dest=0x18ab808, completionTag=0x7ffcf99705a0 "INSERT 0 1") at pquery.c:161
#15 0x00000000009254f0 in PortalRunMulti (portal=0x17d17a8, isTopLevel=true, setHoldSnapshot=false, dest=0x18ab808, altdest=0x18ab808,
completionTag=0x7ffcf99705a0 "INSERT 0 1") at pquery.c:1283
#16 0x0000000000924a4d in PortalRun (portal=0x17d17a8, count=9223372036854775807, isTopLevel=true, run_once=true, dest=0x18ab808, altdest=0x18ab808,
completionTag=0x7ffcf99705a0 "INSERT 0 1") at pquery.c:796
#17 0x000000000091e5b4 in exec_simple_query (query_string=0x1702ad8 "insert into rtest_emp values ('wiecc', '5000.00');") at postgres.c:1215
#18 0x0000000000922a7c in PostgresMain (argc=1, argv=0x1799940, dbname=0x17997b8 "postgres", username=0x1799790 "postgres") at postgres.c:4247
#19 0x00000000008713bd in BackendRun (port=0x17901d0) at postmaster.c:4448
#20 0x0000000000870b06 in BackendStartup (port=0x17901d0) at postmaster.c:4139
#21 0x000000000086cc46 in ServerLoop () at postmaster.c:1704
#22 0x000000000086c46d in PostmasterMain (argc=3, argv=0x16fd750) at postmaster.c:1377
#23 0x00000000007850f7 in main (argc=3, argv=0x16fd750) at main.c:228
```
2) If we have at least one worker node **with metadata sync is disabled** , below insert errors out with the following error report as it tries to insert to `rtest_emplog` (shell table), but it does not exist on worker:
```sql
ERROR: relation "test.rtest_emplog" does not exist
CONTEXT: while executing command on localhost:9701
```
3) If we have at least one worker node **with metadata sync is enabled** , below insert errors out with the following error report:
```sql
WARNING: cannot use 2PC in transactions involving multiple servers
WARNING: connection to the remote node localhost:9702 failed with the following error: another command is already in progress
WARNING: failed to roll back prepared transaction 'citus_0_5929_24_5'
HINT: Run "ROLLBACK PREPARED 'citus_0_5929_24_5'" on localhost:9702
ERROR: cannot use 2PC in transactions involving multiple servers
CONTEXT: while executing command on localhost:9701
```
Contributor guide
Assessment
This issue has not been assessed yet.