electric-sql / electric-sql/electric
Free inactive shape resources to reduce memory usage
- Dominant language
- TypeScript
- Stars
- 10.4k
- Forks
- 375
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 18
Description
We've recently started loading shapes lazily #3243 to avoid a thundering shape creation during boot, to have quicker boot times and more sustainable resource usage growth.
Now, we want to explore freeing shape resources after a period of inactivity. The idea is that this reduces memory usage by maintaining only active shape processes (bounded by shape cache size).
Trade-off: requires opening/closing file handlers during shape matching, which is a critical path operation. This experiment should be evaluated before pursuing pooled shape consumers or other alternatives, as they would have to still manage file handlers.
Contributor guide
Assessment
This issue has not been assessed yet.