pingcap / pingcap/tidb

statistics: TestDropPartitionStats can hit cleanup timeout in CI due to unclosed internal RecordSet

Open
#67,999 2 comments 0 reactions 0 assignees View on GitHub
severity/minor sig/planner type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

Observed on PR `#67814`:
- failing check: `idc-jenkins-ci-tidb/unit-test`
- failing Bazel target: `//pkg/executor/test/simpletest:simpletest_test (shard 8 of 12)`

The failing test is:
- `TestDropPartitionStats`

CI logs show the same pattern across retries:

```text
=== RUN TestDropPartitionStats
...
[db.go:625] ["Waiting for closer"]
--- PASS: TestDropPartitionStats (116.41s / 118.14s / 125.44s)
-- Test timed out --
```

So the test body passes, but the process stalls during cleanup and then hits Bazel timeout.

The suspicious path is:
- `TestDropPartitionStats`
- `flush stats_delta *.*`
- `DumpStatsDeltaToKV(true)`
- `pkg/statistics/handle/storage/update.go:UpdateStatsMeta`

`UpdateStatsMeta` executes two internal `SELECT ... FOR UPDATE` SQLs via `statsutil.ExecWithCtx(...)`, but the returned `RecordSet` is not explicitly closed.

I made a minimal local validation patch that only closes those two internal `RecordSet`s, then reran:

```bash
cd pkg/executor/test/simpletest
go test -race -run TestDropPartitionStats -tags=intest,deadlock -count=20 -timeout=20m
```

Result:
- before patch: `104.291s`
- after patch: `65.068s`

This strongly suggests the unclosed `RecordSet` is delaying cleanup and making CI timeout much more likely.

### 2. What did you expect to see? (Required)

- `TestDropPartitionStats` should finish and clean up normally in CI.
- Internal stats SQL should not leave resources that delay `domain close` / `store close`.

### 3. What did you see instead (Required)

- `TestDropPartitionStats` passes its assertions, but cleanup stalls at badger `Waiting for closer`.
- The same timeout pattern happens repeatedly in CI.
- Explicitly closing the internal `RecordSet` significantly reduces local runtime.

### 4. What is your TiDB version? (Required)

Observed on PR `#67814`:
- commit: `4965430ff5c6d779d798ca62d550948da5b44e65`

Related context:
- suspected root-cause PR: `#58791`
- test-path changes: `#65870`, `#67728`

Suggested labels:
- `type/bug`
- `type/regression`
- `component/statistics`

Contributor guide

Open the contributing guide

Research direction

Start with TestDropPartitionStats in pkg/executor/test/simpletest and trace the flush stats_delta path into pkg/statistics/handle/storage/update.go:UpdateStatsMeta. Run the provided race-enabled test command and compare cleanup behavior; done means the test completes without a cleanup timeout and internal stats SQL resources no longer delay domain or store closure.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
database, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.