cockroachdb / cockroachdb/cockroach
schemachanger: reduce memory usage
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
We should reduce our memory usage for the schema changer to avoid the possibility of OOM. There are a few major sources of memory usage:
1) The `scpb.Target` (because it embeds `scpb.ElementProto`) is very large. This is because `gogoproto.onlyone` makes a field for each variant. We should switch to using `oneof`, which will add an extra indirection, but reduce the total footprint of the object by a large factor. The challenge of this is that we need to teach rel how to indirect through the generated intermediate struct, but to not index it. This isn't hard, but it is a new feature.
2) We should look at the size of the rel graph, and, in particular, the byte size of interning the strings and objects. Maybe the interface boxes could be `unsafe.Pointer`s which would reduce their footprint by a factor of 2, and, perhaps, the indexes could use `int32` instead of `int` to reduce their size by a factor of 2.
Maybe also, the `graph` itself is big, and the nodes and objects there could be made much more compact.
This issue is less pressing than https://github.com/cockroachdb/cockroach/issues/98631, but it is legitimately important.
Jira issue: CRDB-25378
Epic CRDB-60938
Contributor guide
Assessment
This issue has not been assessed yet.