cockroachdb / cockroachdb/cockroach
sql/opt: no memory accounting or budget for planning
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Planning-time allocations -- memo, interner, metadata, logical props, constraint spans -- are plain Go heap, invisible to the SQL memory monitor hierarchy. Nothing in pkg/sql/opt takes a `mon.BoundAccount`, so `--max-sql-memory` never engages while a statement is being planned; the only backstop is GOMEMLIMIT / the OOM killer. The only accounting today is post-hoc:
`MemoryEstimate` is charged for retained memos of prepared statements and the query plan cache -- final size only, after planning has already succeeded.
We should bound optimizer memory usage generally, so that a statement whose planning exceeds the budget fails with a clear error.
----
Related:
Several shape-specific caps were added as mitigation for OOM issues that we've seen during planning: `reorder_joins_limit`, `optimizer_span_limit` (#167620), and `optimizer_max_disjunction_split_count` (#172644, from a support escalation). The next unbounded shape will bypass memory monitoring the same way.
Jira issue: CRDB-66031
Contributor guide
Research direction
Start by tracing planning-time allocations under pkg/sql/opt and the existing post-hoc MemoryEstimate charging for prepared statements and the query plan cache. Compare those paths with mon.BoundAccount and the --max-sql-memory behavior. Done means optimizer planning is generally budgeted and a statement that exceeds the budget fails with a clear error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100