citusdata / citusdata/citus

Consider not caching connections on maintenance daemon

Open
#3,671 0 comments 0 reactions 0 assignees View on GitHub
mx
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

Currently, Citus caches `citus.max_cached_conns_per_worker` on the maintenance daemon as well. To see that, query `pg_stat_activity` on any worker and you'd see `dump_local_wait_edges()` in an idle state (most of the time)

```SQL
select query,state, usename from pg_stat_activity WHERE backend_type = 'client backend';
┌──────────────────────────────────────────────────────────────────────────────────────────┬────────┬─────────────┐
│ query │ state │ usename │
├──────────────────────────────────────────────────────────────────────────────────────────┼────────┼─────────────┤
│ select query,state, usename from pg_stat_activity WHERE backend_type = 'client backend'; │ active │ onderkalaci │
│ SELECT * FROM dump_local_wait_edges() │ idle │ onderkalaci │
└──────────────────────────────────────────────────────────────────────────────────────────┴────────┴─────────────┘
(2 rows)

Time: 11.820 ms
```

Some things to consider:
- This might be OK, but it'd be good to evaluate this further. Especially with function propagation, we'd be syncing metadata to the workers, meaning that each worker would cache one connection to the others.
- The connection used is a superuser connection as it is established with the extension owner user. It can use lots of reserved superuser connections, which we wouldn't want.
- Hypercale(Citus) currently logs all connection establishments. So, re-connecting every time might lead lots of logs that are almost unnecessary.
- The performance effect of re-connecting every 2 seconds should be evaluated.

We could potentially add something like the following on the maintenance daemon right after every `StartTransactionCommand`:
```
StartTransactionCommand();

set_config_option("citus.max_cached_conns_per_worker", "0",
(superuser() ? PGC_SUSET : PGC_USERSET), PGC_S_SESSION,
GUC_ACTION_LOCAL, true, 0, false);
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.