planner: optimizer range building memory is not tracked for long IN lists
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
This is an observability and memory-attribution issue in the planner.
A minimal scenario:
1. Enable OOM/top SQL memory diagnostics, or inspect SQL/session memory through the existing memory tracker based diagnostics.
2. Run a statement with a very long `IN` list on an indexed column, for example:
```sql
CREATE TABLE t (a VARCHAR(64), b INT, KEY idx_a(a));
EXPLAIN SELECT * FROM t WHERE a IN (... tens or hundreds of thousands of constants ...);
```
3. Observe memory growth while the optimizer builds ranges for the long `IN` list.
Code evidence from current master:
- `RangerContext` does not carry a query/session `MemTracker`: `pkg/util/ranger/context/context.go`.
- `builder.buildFromIn` allocates range points proportional to `len(list) * 2`, copies datum values, then sorts/deduplicates them without calling `MemTracker.Consume`: `pkg/util/ranger/points.go`.
- `points2Ranges` estimates memory only for `tidb_opt_range_max_size` fallback; it does not account the allocation to the SQL memory tracker: `pkg/util/ranger/ranger.go`.
- OOM/top SQL memory attribution depends on `Tracker.Consume` and `ProcessInfo.MemTracker.MaxConsumed()`: `pkg/util/memory/tracker.go`, `pkg/util/memoryusagealarm/memoryusagealarm.go`.
- The cached-plan range rebuild path calls ranger with `rangeMaxSize = 0`, so the normal range-size fallback is not applied there: `pkg/planner/core/plan_cache_rebuild.go`.
### 2. What did you expect to see? (Required)
Transient memory allocated while building optimizer ranges for long `IN` lists should be attributed to the current SQL/session memory tracker, or otherwise be visible in OOM/top SQL diagnostics.
`tidb_opt_range_max_size` fallback and plan-cache range rebuilding should not leave an observability blind spot where the memory is consumed by the current SQL but cannot be traced back to it by TiDB diagnostics.
### 3. What did you see instead (Required)
The memory used during optimizer range-point/range construction is not consistently recorded by the SQL/session `MemTracker`.
As a result, when a long `IN` list causes high transient memory usage during optimization, OOM alarm/top SQL diagnostics can miss or underreport the SQL responsible. `tidb_opt_range_max_size` may prevent some range explosion cases by falling back, but that is separate from memory attribution, and the cached-plan range rebuild path currently bypasses that limit by passing `rangeMaxSize = 0`.
### 4. What is your TiDB version? (Required)
Observed from current master code inspection on 2026-04-16. Exact affected released versions have not been verified.
### Additional context
Related but not a duplicate: #37176 tracks the broader memory-control/refactor problem for building ranges. This issue is specifically about memory tracking and OOM/top SQL attribution for optimizer range building with long `IN` lists.
Suggested labels: `type/bug`, `sig/planner`, `component/planner`, `severity/moderate`, `may-affects-master`
Contributor guide
Assessment
This issue has not been assessed yet.