apache / apache/cloudberry

Replace the hacky way stats pg_appendonly.segfilecount

Open
#97 1 comment 0 reactions 0 assignees View on GitHub
type: Enhancement
Dominant language
C
Stars
1.4k
Forks
247
Avg merge
4d 3h
Merged PRs (30d)
39

Description

### Cloudberry Database version

_No response_

### What happened

This is added by me when implementing AO/AOCS's parallel scan feature.

`segfilecount` is used to parallel workers for AO/AOCS.
It's updated by VACCUM, ANALYZE, TRUNCATE command and auto vacuum process.
I use VPgClassStats.relallvisible to represent segment file count of AO/AOCS as we are hurry for that feature, and I could
save a MessageType of libpq and etc.

It works well because AO/AOCS does not currently have an equivalent to Heap's 'all visible pages',
relallvisible is always set to 0 in pg_class for AO/AOCS tabes. But QE use this field in libpq to represent AO/AOCS's total file segment number when vacuum ao tables.
See more comments in below files.

```c
typedef struct VPgClassStats
{
Oid relid;
BlockNumber rel_pages;
double rel_tuples;
BlockNumber relallvisible;
} VPgClassStats;
```
The approach is very hacky and not good.
We should reconsider the implementation for this including VACUUM, ANALYZE, TRUNCATE and auto vacuum process.

vacuum_ao.c
```c
/* AO/AOCO does not currently have an equivalent to
Heap's 'all visible pages', use this field to represent
AO/AOCO's total segment file count */
```
vacuum.h
```c
* relallvisible
* AO/AOCO does not currently have an equivalent to Heap's 'all visible pages',
* relallvisible is always set to 0 in pg_class for AO/AOCO tabes. But QE use
* this field in libpq to represent AO/AOCO's total file segment number when
* vacuum ao tables.
* Remember to reset relallvisible to 0 after qd get file segment number and
* before updating pg_class.
* See vac_update_relstats_from_list in vacuum.c and
* vac_update_relstats in vacuum_ao.c
```

### What you think should happen instead

_No response_

### How to reproduce

No need.

### Operating System

Ubuntu.

### Anything else

_No response_

### Are you willing to submit PR?

- [ ] Yes, I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://cloudberrydb.org/community/coc).

Contributor guide

Open the contributing guide

Research direction

Start by reading the comments and relallvisible handling in vacuum_ao.c and vacuum.h, then trace how VACUUM, ANALYZE, TRUNCATE, and autovacuum update the value and how QE consumes it through libpq. Done means the segment-file count no longer uses relallvisible and the affected maintenance paths and consumer agree on a dedicated representation.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.