cockroachdb / cockroachdb/cockroach

sql: Memory accounting for compiled regular expressions

Open
#94,000 0 comments 0 reactions 0 assignees View on GitHub
C-enhancement T-sql-queries
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

The regexp functions can consume very large amounts of memory, but this is not tracked in our memory accounting. In Go, the memory footprint of a compiled regular expression is O(n) in the length of the input string (which *is* accounted for correctly, perhaps leading us to think that tracking the compiled regexp object is unnecessary), but the constant can be surprisingly high. A compiled regexp can be as much as 40,000 times the size of its input string. We need to account for this memory so that regular expressions do not become a vector for a query to consume excessive memory and potentially crash the server.

This is related to CVE-2022-41715, fixed in Go 1.19.2. However, the fix in Go 1.19.2 is insufficient for our purposes and we still need to improve our accounting in this area. That patch imposes a limit on how big a *single* compiled regexp could become, but we still need proper accounting so that a query that compiles *many* regular expressions cannot consume too much memory.

This accounting is made more complicated by the fact that we want to cache compiled regexp objects; we need to make sure we do not unknowingly hold onto very large regexps here.

Jira issue: CRDB-22643

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.