cockroachdb / cockroachdb/cockroach

sql: stats collection doesn't get re-tried after a server shutdown

Open
#100,482 4 comments 0 reactions 0 assignees View on GitHub
A-sql-table-stats C-bug T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Discovered by @rytaft

**Describe the problem**

The automatic stats collection (used by SQL query planning) is triggered lazily when tables encounter mutations.

This lazy mechanism is based on an in-memory FIFO (a Go channel).

If a server gets shut down *after* a SQL mutation completes but *before* the mutation was popped from this FIFO (and thus before the job to compute the stats has been created), there will never be any re-computation.

This can result in bad query plans for tables that had a bunch of changes just before a node shuts down and then never change again.

**To Reproduce**

Issue a large number of mutations to a table and issue a server shutdown in the middle of these mutations.
Then restart the server and do not modify the table again. Notice that the stats remain out of date forever.

**Expected behavior**

- Either persist the in-flight FIFO queue of mutated tables so they get a chance to see their jobs created when the server starts again.
- Wait during graceful shutdown until all the jobs are created, but without running them. (This could delay the drain in an unacceptable way if there were many tables. Testing should evaluate this overhead)

**Environment:**

crdb v23.1 and `master` branch

Jira issue: CRDB-26459

Contributor guide

Open the contributing guide

Research direction

Start by tracing automatic stats collection from the in-memory FIFO through server shutdown and restart, using the mutation-and-shutdown reproduction described in the issue. Determine how to ensure a table mutated before shutdown is reconsidered after restart, and validate that its statistics are no longer permanently out of date without further mutations.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.