Make useful postgresql monitoring views avaliable on the coordinator per shard/distributed table
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
PostgreSQL has quite a lot of useful [views](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-STATS-VIEWS-TABLE) that are becoming very hard for Citus users to utilize.
The primary reason for these views not being useful is that the data resides on the shards, and the views are mostly related with the actual data.
For example, `pg_stat_all_indexes` shows super interesting information about indexes, but that's not useful for distributed tables, because the actual statistics should be pulled from the shards.
For example, check the number of index scans:
```SQL
-- the coordinator has no valuable information
SELECT idx_scan FROM pg_stat_all_indexes where relname = 'users_table';
┌─[ RECORD 1 ]─┐
│ idx_scan │ 0 │
└──────────┴───┘
-- but this is a lot more useful
SELECT run_command_on_workers($$SELECT sum(idx_scan) FROM pg_stat_all_indexes where relname LIKE 'users_table_%'$$);
┌─[ RECORD 1 ]───────────┬───────────────────────┐
│ run_command_on_workers │ (localhost,9700,t,12) │
├─[ RECORD 2 ]───────────┼───────────────────────┤
│ run_command_on_workers │ (localhost,9701,t,7) │
├─[ RECORD 3 ]───────────┼───────────────────────┤
│ run_command_on_workers │ (localhost,9702,t,1) │
└────────────────────────┴───────────────────────┘
```
We should aim to have a generic way of pulling these kind of useful information to the coordinator.
I think, keeping these statistics per shard on the coordinator could be super useful (and on top a global/aggregated view per distributed table as well).
We can utilize these types of views for (a) giving very useful insights about the cluster (b) use for distributed planning.
Note that we've a similar approach implemented in `citus_dist_stat_activity`, which I find very useful in general.
(Thanks @DimCitus for bringing this idea).
Contributor guide
Assessment
This issue has not been assessed yet.