Graylog2 / Graylog2/graylog2-server
ClusterMetricsResource hard timeout for big clusters
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
We have a customer with very big cluster consisting of some 160 graylog nodes. The ClusterMetricsResource is constantly failing to deliver responses, because there is a hard limit configured:
```
Duration.ofMillis(Math.min(defaultCallTimeout.toMilliseconds(), 1000))
```
the nodes are all responding, but the 1000ms hard limit is just to strict to collect all metrics.
## Expected Behavior
ClusterMetricsResource doesn't fail for larger installations
## Possible Solution
Either make the 1000ms configurable as well (and strongly warn against changing that, because it can lead to exhausting thread pools elsewhere) or let the nodes send their metrics regularly, persist them and let ClusterMetricsResource just aggregate these results.
Contributor guide
Assessment
This issue has not been assessed yet.