cockroachdb / cockroachdb/cockroach

sql: limit stats forecast timestamp when there are large gaps between stats collections

Open
#117,890 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement O-support P-3 T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

See https://github.com/cockroachlabs/support/issues/2792

To determine the date to forecast table statistics at we add the average interval between stats collections to the last collection date. This works well in many cases, but causes problems when the are large gaps between stats collections.

Imagine we have collections at `t1`, `t2`, `t3`, `t60`, and `t61`, where `tX - tY` is proportional to `X - Y`, for all `tN`. The average time between collections is 15, so we'll forecast out to `t76`. This can be fairly far in the future, in which case the forecast won't be representative of the data currently in the table. This can ultimately cause bad query plans.

Some ideas for addressing this:

1. Ignore outliers in the time between collections when calculating the forecast time.
2. Cap the forecast time to be at most some point in the future. We've previously tried to set this to a week but we found forecasting to be unaffective in some real-world cases.

Jira issue: CRDB-35405

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.