Altinity / Altinity/clickhouse-operator
metric DiskDataBytes can be incorrect due to inactive parts & hardlinks
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.6k
- Forks
- 574
- Avg merge
- 8d 6h
- Merged PRs (30d)
- 6
Description
After mutation (especially noop mutation) the whole dataset becomes inactive, and hardlink-copy of that becomes active.
We count both active & inactive in one single metric
https://github.com/Altinity/clickhouse-operator/blob/b0dc549109edd58a47005a4118160de9c23bb606/pkg/apis/metrics/clickhouse_metrics_fetcher.go#L96
https://github.com/Altinity/clickhouse-operator/blob/b0dc549109edd58a47005a4118160de9c23bb606/pkg/apis/metrics/prometheus_writer.go#L139
and that may look like the table consumes twice the disk space.
I would count activeparts disk space and inactive parts disk space separately.
Also that metric does not account temp files & detached parts and non-mergetree data etc (so the name is not the best).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the metric assembly in pkg/apis/metrics/clickhouse_metrics_fetcher.go around line 96 and output handling in pkg/apis/metrics/prometheus_writer.go around line 139. Trace how active and inactive parts are combined, then define the expected separation and naming for those values, including whether temp, detached, and non-MergeTree data are in scope. Done means the reported disk metrics no longer double-count hardlinked inactive data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, prometheus
- Domain
- databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100