cockroachdb / cockroachdb/cockroach
sql: limit stats forecast timestamp when there are large gaps between stats collections
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
See https://github.com/cockroachlabs/support/issues/2792
To determine the date to forecast table statistics at we add the average interval between stats collections to the last collection date. This works well in many cases, but causes problems when the are large gaps between stats collections.
Imagine we have collections at `t1`, `t2`, `t3`, `t60`, and `t61`, where `tX - tY` is proportional to `X - Y`, for all `tN`. The average time between collections is 15, so we'll forecast out to `t76`. This can be fairly far in the future, in which case the forecast won't be representative of the data currently in the table. This can ultimately cause bad query plans.
Some ideas for addressing this:
1. Ignore outliers in the time between collections when calculating the forecast time.
2. Cap the forecast time to be at most some point in the future. We've previously tried to set this to a week but we found forecasting to be unaffective in some real-world cases.
Jira issue: CRDB-35405
Contributor guide
Assessment
This issue has not been assessed yet.