cockroachdb / cockroachdb/cockroach

sql/opt: no memory accounting or budget for planning

Open
#172,768 2 comments 0 reactions 0 assignees View on GitHub
A-sql-memmon A-sql-optimizer C-enhancement O-support P-2 T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Planning-time allocations -- memo, interner, metadata, logical props, constraint spans -- are plain Go heap, invisible to the SQL memory monitor hierarchy. Nothing in pkg/sql/opt takes a `mon.BoundAccount`, so `--max-sql-memory` never engages while a statement is being planned; the only backstop is GOMEMLIMIT / the OOM killer. The only accounting today is post-hoc:
`MemoryEstimate` is charged for retained memos of prepared statements and the query plan cache -- final size only, after planning has already succeeded.

We should bound optimizer memory usage generally, so that a statement whose planning exceeds the budget fails with a clear error.

----

Related:
Several shape-specific caps were added as mitigation for OOM issues that we've seen during planning: `reorder_joins_limit`, `optimizer_span_limit` (#167620), and `optimizer_max_disjunction_split_count` (#172644, from a support escalation). The next unbounded shape will bypass memory monitoring the same way.

Jira issue: CRDB-66031

Contributor guide

Open the contributing guide

Research direction

Start by tracing planning-time allocations under pkg/sql/opt and the existing post-hoc MemoryEstimate charging for prepared statements and the query plan cache. Compare those paths with mon.BoundAccount and the --max-sql-memory behavior. Done means optimizer planning is generally budgeted and a statement that exceeds the budget fails with a clear error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.