pingcap / pingcap/tidb

executor: missing Close() on MemBuffer snapshot iterators can cause memory growth

Open
#65,514 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

### 1. Minimal reproduce step (Required)

> This issue is most visible after TiDB switches to client-go's ART-based MemDB.

1. Use a TiDB build/version that uses client-go with ART MemDB.
2. Run a workload that repeatedly triggers union scan over in-transaction MemBuffer data.

Example SQL (simplified):

```sql
CREATE TABLE t (
a INT PRIMARY KEY,
b INT,
KEY idx_b(b)
);

INSERT INTO t VALUES (1,1),(2,2),(3,3);
ANALYZE TABLE t;

-- Repeat this transaction many times (e.g. hundreds/thousands):
BEGIN;
INSERT INTO t VALUES (100, 100) ON DUPLICATE KEY UPDATE b = VALUES(b);
SELECT /* force union scan */ a,b FROM t USE INDEX(idx_b) WHERE b >= 0 ORDER BY b LIMIT 10;
COMMIT;
```

3. Observe TiDB process memory (e.g. `top`) or take heap profiles.

### 2. What did you expect to see? (Required)

Memory usage should remain stable (or at least not grow unbounded) under a steady repeating workload.

### 3. What did you see instead? (Required)

The TiDB process memory keeps increasing across repeated executions, and does not drop after transactions complete.

### 4. What is your TiDB version? (Required)

- TiDB: master (observed after client-go ART memdb adoption)
- client-go: `github.com/tikv/client-go/v2 v2.0.8-0.20251215121014-01758810e841`

### 5. Root cause analysis (Optional)

Some code paths in `pkg/executor/mem_reader.go` create MemBuffer snapshot iterators via `txn.GetMemBuffer().SnapshotIter()` / `SnapshotIterReverse()` but do not always call `Close()`.

With client-go's ART MemDB implementation, snapshot iterators keep a snapshot ref-count > 0 until `Close()` is called. While the ref-count is non-zero, old-version nodes cannot be freed/reused, which may lead to continuous memory growth under repeated workloads.

### 6. Proposed fix

Ensure every iterator returned by `SnapshotIter` / `SnapshotIterReverse` is properly closed:
- on normal completion
- when switching ranges (close the previous iterator)
- on all error paths

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.