pingcap / pingcap/tidb

CI unit tests can take hours due to slow/flaky testcases

Open
#65,600 0 comments 0 reactions 0 assignees View on GitHub
type/performance
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

Background

TiDB unit tests (UT) in CI can take up to ~3 hours (e.g. pull_unit_test_next_gen), where a non-trivial part is caused by a few very slow testcases and flake amplifiers (timeouts / fixed sleeps / resource conflicts).

Local evidence (tools/bin/ut + JUnit timing)

- nextgen baseline: 51 testcases >10s, 3 testcases hit 10m timeout
- legacy baseline: 21 testcases >10s

Goal

- Reduce/avoid single very slow testcases (e.g. >10s) and remove flake amplifiers, while keeping coverage and deterministic behavior.
- Long term: drive toward <3s per testcase where possible.

Approach

- Profile local full UT runs via tools/bin/ut JUnit timing.
- Optimize offenders by reducing data volume / loops, replacing fixed sleeps with bounded waits, gating stress cases behind testflag.Long(), and fixing race windows that cause flakes/timeouts.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.