pingcap / pingcap/tidb

Concurrent SET SNAPSHOT may trigger duplicated schema full load and cause OOM on schema cache miss

Open
#65,181 0 comments 0 reactions 0 assignees View on GitHub
component/ddl may-affects-7.1 may-affects-7.5 may-affects-8.1 may-affects-8.5 severity/major type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

When **`SET SNAPSHOT` is executed concurrently** and the **target snapshot schema version is not cached**, TiDB may trigger **concurrent schema full loads**, leading to **rapid memory allocation and OOM**, which is unexpected.

In one production cluster (**Cluster A**), a reconciliation job concurrently executed `SET SNAPSHOT` with a fixed snapshot time (`2025-12-22 11:20`). Since the corresponding historical schema version was **not present in the schema cache**, each request entered the schema full load path. As a result, multiple goroutines concurrently invoked `fetchSchemasWithTables`, causing memory usage to spike. The goroutine dump shows **173 identical `fetchSchemasWithTables` call stacks**, and TiDB eventually OOMed.

The cache miss was caused by rapid schema version advancement after the snapshot time (18 DDLs between 11:20 and 11:25), while `tidb_schema_version_cache_limit` was set to 16, which did not cover the required historical schema version.

In another production cluster (**Cluster B**), no DDL occurred after the snapshot time, so the required schema version remained cached. Even with concurrent `SET SNAPSHOT`, the cached infoschema was reused, no schema full load was triggered, and CPU and memory usage remained normal.

Image

### 2. What did you expect to see? (Required)
No OOM

### 3. What did you see instead (Required)
tidb-server OOM

### 4. What is your TiDB version? (Required)
This issue was observed on TiDB v7.5.6.
Since the behavior is related to schema cache miss and concurrent schema full load, it is expected to affect all TiDB versions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.