cockroachdb / cockroachdb/cockroach

schemachanger: reduce memory usage

Open
#98,633 2 comments 0 reactions 0 assignees View on GitHub
C-enhancement C-investigation T-sql-foundations
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

We should reduce our memory usage for the schema changer to avoid the possibility of OOM. There are a few major sources of memory usage:

1) The `scpb.Target` (because it embeds `scpb.ElementProto`) is very large. This is because `gogoproto.onlyone` makes a field for each variant. We should switch to using `oneof`, which will add an extra indirection, but reduce the total footprint of the object by a large factor. The challenge of this is that we need to teach rel how to indirect through the generated intermediate struct, but to not index it. This isn't hard, but it is a new feature.

2) We should look at the size of the rel graph, and, in particular, the byte size of interning the strings and objects. Maybe the interface boxes could be `unsafe.Pointer`s which would reduce their footprint by a factor of 2, and, perhaps, the indexes could use `int32` instead of `int` to reduce their size by a factor of 2.

Maybe also, the `graph` itself is big, and the nodes and objects there could be made much more compact.

This issue is less pressing than https://github.com/cockroachdb/cockroach/issues/98631, but it is legitimately important.

Jira issue: CRDB-25378

Epic CRDB-60938

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.