cockroachdb / cockroachdb/cockroach
sql: Memory accounting for compiled regular expressions
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
The regexp functions can consume very large amounts of memory, but this is not tracked in our memory accounting. In Go, the memory footprint of a compiled regular expression is O(n) in the length of the input string (which *is* accounted for correctly, perhaps leading us to think that tracking the compiled regexp object is unnecessary), but the constant can be surprisingly high. A compiled regexp can be as much as 40,000 times the size of its input string. We need to account for this memory so that regular expressions do not become a vector for a query to consume excessive memory and potentially crash the server.
This is related to CVE-2022-41715, fixed in Go 1.19.2. However, the fix in Go 1.19.2 is insufficient for our purposes and we still need to improve our accounting in this area. That patch imposes a limit on how big a *single* compiled regexp could become, but we still need proper accounting so that a query that compiles *many* regular expressions cannot consume too much memory.
This accounting is made more complicated by the fact that we want to cache compiled regexp objects; we need to make sure we do not unknowingly hold onto very large regexps here.
Jira issue: CRDB-22643
Contributor guide
Assessment
This issue has not been assessed yet.