Improve dynamic sizing of internal buffers
- Dominant language
- Go
- Stars
- 1.3k
- Forks
- 543
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 109
Description
Following up from #9358, we discussed that it would be good to take the actual `config.MaxRequestSize` and the `ouptutConfig.FlushBytes` values rather than assume that they are set to the default values to avoid situations where users may increase those buffers to higher values.
We are currently sizing 3 settings based on the amount of RAM available (80% for cgroup and 50% for non-cgroup servers):
- Semaphore: `128 * mem`.
- Internal modelindexer queue: `1024 * mem`.
- Available indexers: `10 + mem * 1.5`.
`config.MaxRequestSize` should be considered for the semaphore and the internal queue, while `ouptutConfig.FlushBytes` should be considered for the number of available indexers.
Contributor guide
Assessment
This issue has not been assessed yet.