cockroachdb / cockroachdb/cockroach

sql: memory exceeded in sampleAggregator causes stats collection to fail

Open
#127,649 1 comment 0 reactions 0 assignees View on GitHub
A-sql-table-stats branch-master C-bug O-qa T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

As part of table statistics collection , there is logic in the `sampler` and `sampleAggregator` to reduce the number of histogram samples collected or disable histogram collection if the memory budget is exceeded. However, it seems that for certain sample sizes and bucket counts, exceeding the memory budget causes stats collection to fail altogether.

**To Reproduce**

1. Start a single node cockroach cluster and connect to the sql shell:
```
~/Downloads/cockroach-v24.2.0-beta.1.darwin-11.0-arm64/cockroach start-single-node --insecure

# on a different screen:
~/Downloads/cockroach-v24.2.0-beta.1.darwin-11.0-arm64/cockroach sql --insecure
```

2. Run the following:
```
SET CLUSTER SETTING sql.stats.automatic_collection.enabled = 'false';
SET CLUSTER SETTING sql.stats.histogram_buckets.count = 500000;
CREATE TABLE tab (a INT, b STRING, c TIMESTAMP, INDEX (a), INDEX(b), INDEX(c));
INSERT INTO tab SELECT i, i::string, now() + i::interval FROM generate_series(1, 500000) AS g(i);
ALTER TABLE tab INJECT STATISTICS '[
{
"avg_size": 2,
"columns": [
"a"
],
"created_at": "2024-07-23 16:53:43.380334",
"distinct_count": 800000000,
"histo_version": 3,
"null_count": 0,
"row_count": 800000000
}
]';
ANALYZE tab;
```

The `ANALYZE` should fail with the error `ERROR: sample-aggregator-mem: memory budget exceeded: 3758080 bytes requested, 65085440 currently allocated, 0 bytes in budget`.

**Expected behavior**

Instead of causing an error and failing to collect stats, we should reduce the number of samples/buckets and/or disable histogram collection.

**Environment:**
- CockroachDB version 24.2 beta
- Server OS: MacOS ARM
- Client app: `cockroach sql`

Jira issue: CRDB-40535

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.