cockroachdb / cockroachdb/cockroach
sql/stats: prevent "infinite loop" with "fix misestimates" automatic stats collection
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
In the original implementation of the "fix misestimates" auto stats collection (trigger partial stats on the spans based on the feedback from the execution engine, #152995) we have a failure mode in which we can be continuously collecting "fix misestimates" stats without actually fixing misestimates. This [thread](https://cockroachlabs.slack.com/archives/G01R3EYPQKA/p1757600998044749) contains nice discussion of the problem and some approaches to the solution, copying Becca's problem description:
> If we're not careful, we could run into the following failure mode: say we have 2 columns in an index, `(cust_id, date)`. Our stats on `cust_id` are actually up to date, but our stats on `date` are stale, because `date` is an ascending key and by definition will always be somewhat stale. When we have a query that constrains both columns and see that the actual number of rows doesn't match the expected number, we'll trigger a partial stats collection. Our current plan is to only update the histogram for `cust_id` based on the partial stats collection, since that's the only column for which we can be confident in our new stats. However, that won't solve the issue, since it was really the stats on `date` that were stale. As a result, our execution engine will continue to trigger new partial stats collections that are basically useless.
Jira issue: CRDB-61300
Contributor guide
Assessment
This issue has not been assessed yet.