module memberlist-kv has failed immediately on 240.0.0.0/4 subnet
- Dominant language
- Go
- Stars
- 11.7k
- Forks
- 802
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 80
Description
#### Describe the bug
In the k8s infra, we use 240.0.0.0/4 for the pod ips and pyroscope has failed to start with the log output:
```
ts=2024-02-09T04:35:06.888387925Z caller=server.go:353 level=info msg="server listening on addresses" http=[::]:4040 grpc=[::]:9095
ts=2024-02-09T04:35:06.901229425Z caller=memberlist_client.go:435 level=info msg="Using memberlist cluster label and node name" cluster_label= node=efc8384c61f2-02366467
ts=2024-02-09T04:35:06.902561258Z caller=modules.go:526 level=warn msg="no storage bucket configured, the admin component will not be loaded"
ts=2024-02-09T04:35:06.902881591Z caller=modules.go:146 level=warn msg="using in-memory settings store, changes will be lost after shutdown"
ts=2024-02-09T04:35:06.912083633Z caller=netutil.go:90 level=warn component=overrides-exporter msg="error getting addresses for interface" inf=en0 err="route ip+net: no such network interface"
ts=2024-02-09T04:35:06.9134403Z caller=worker.go:113 level=info component=querier-worker msg="Starting querier worker connected to query-scheduler" scheduler=
ts=2024-02-09T04:35:06.921183091Z caller=module_service.go:82 level=info msg=starting module=server
ts=2024-02-09T04:35:06.923611383Z caller=module_service.go:82 level=info msg=starting module=memberlist-kv
ts=2024-02-09T04:35:06.923655175Z caller=module_service.go:82 level=info msg=starting module=tenant-settings
ts=2024-02-09T04:35:06.925901716Z caller=phlare.go:423 level=error msg="module failed" module=memberlist-kv error="starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.9259388Z caller=phlare.go:423 level=error msg="module failed" module=query-frontend error="failed to start query-frontend, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.925947716Z caller=phlare.go:423 level=error msg="module failed" module=usage-stats error="failed to start usage-stats, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.925953216Z caller=phlare.go:423 level=error msg="module failed" module=overrides-exporter error="failed to start overrides-exporter, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.9259588Z caller=phlare.go:423 level=error msg="module failed" module=query-scheduler error="failed to start query-scheduler, because it depends on module usage-stats, which has failed: invalid service state: Failed, expected: Running, failure: failed to start usage-stats, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.925967883Z caller=phlare.go:423 level=error msg="module failed" module=distributor error="failed to start distributor, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.92598355Z caller=phlare.go:423 level=error msg="module failed" module=version error="failed to start version, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.925993675Z caller=phlare.go:423 level=error msg="module failed" module=ring error="failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.926004133Z caller=phlare.go:423 level=error msg="module failed" module=querier error="failed to start querier, because it depends on module ring, which has failed: invalid service state: Failed, expected: Running, failure: failed to start ring, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.926015258Z caller=phlare.go:423 level=error msg="module failed" module=ingester error="failed to start ingester, because it depends on module memberlist-kv, which has failed: invalid service state: Failed, expected: Running, failure: starting module memberlist-kv: invalid service state: Stopping, expected: Running"
ts=2024-02-09T04:35:06.926829841Z caller=module_service.go:120 level=info msg="module stopped" module=tenant-settings
ts=2024-02-09T04:35:06.927568758Z caller=modules.go:694 level=info msg="server stopped"
ts=2024-02-09T04:35:06.927627383Z caller=module_service.go:120 level=info msg="module stopped" module=server
ts=2024-02-09T04:35:06.927695883Z caller=phlare.go:499 level=info msg="Pyroscope stopped"
failed running pyroscope: failed services
```
#### To Reproduce
Steps to reproduce the behavior:
1. change subnet in the docker to 240.0.0.0/16
2. docker run -ti --rm grafana/pyroscope:1.3.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.