electric-sql / electric-sql/electric

Free inactive shape resources to reduce memory usage

Open
#3,368 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
10.4k
Forks
375
Avg merge
3d 1h
Merged PRs (30d)
18

Description

We've recently started loading shapes lazily #3243 to avoid a thundering shape creation during boot, to have quicker boot times and more sustainable resource usage growth.

Now, we want to explore freeing shape resources after a period of inactivity. The idea is that this reduces memory usage by maintaining only active shape processes (bounded by shape cache size).
Trade-off: requires opening/closing file handlers during shape matching, which is a critical path operation. This experiment should be evaluated before pursuing pooled shape consumers or other alternatives, as they would have to still manage file handlers.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.