kubernetes-sigs / kubernetes-sigs/cluster-api
Issues in kube-state-metrics dev-setup
- Dominant language
- Go
- Stars
- 4.3k
- Forks
- 1.6k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 113
Description
I saw a few (potential) issues with kube-state-metrics when running our dev-setup. Not sure if they are upstream issues or if they are related to our configuration.
1. Custom resource state added metrics is logged twice for each metric
For example:
``` plain
I0703 11:33:29.295822 1 custom_resource_metrics.go:79] "Custom resource state added metrics" familyNames=[capi_clusterclass_info capi_clusterclass_created capi_clusterclass_annotation_paused capi_clusterclass_status_condition capi_clusterclass_status_condition_last_transition_time capi_clusterclass_owner]
I0703 11:33:29.295883 1 custom_resource_metrics.go:79] "Custom resource state added metrics" familyNames=[capi_clusterclass_info capi_clusterclass_created capi_clusterclass_annotation_paused capi_clusterclass_status_condition capi_clusterclass_status_condition_last_transition_time capi_clusterclass_owner]
```
2. There are a lot of errors/warnings (?) in the kube-state-metric logs e.g. when creating a cluster
```plain
E0703 11:37:51.841041 1 registry_factory.go:649] "capi_kubeadmconfig_status_condition" err="[status,conditions]: expected value for path to be string, got "
E0703 11:37:51.841064 1 registry_factory.go:649] "capi_kubeadmconfig_status_condition_last_transition_time" err="[status,conditions]: got nil while resolving path"
E0703 11:37:51.841081 1 registry_factory.go:649] "capi_kubeadmconfig_status_condition" err="[status,conditions]: expected value for path to be string, got "
E0703 11:37:51.841096 1 registry_factory.go:649] "capi_kubeadmconfig_status_condition_last_transition_time" err="[status,conditions]: got nil while resolving path"
```
Not sure if they are expected. I think this would become very noisy with a few hundreds of clusters.
3. Metrics dont' work with an empy labelsFromPath. I had to work around this in a few cases because we didn't have any metric-specific labels otherwise, e.g.: https://github.com/kubernetes-sigs/cluster-api/blob/39b2431b11863456818837fc5416cba75d99ecd7/hack/observability/kube-state-metrics/crd-config.yaml#L25C1-L26
Contributor guide
Research direction
Start by reproducing the kube-state-metrics behavior in the dev-setup and inspect hack/observability/kube-state-metrics/crd-config.yaml, especially the empty labelsFromPath case. Use the logged custom_resource_metrics.go and registry_factory.go locations to separate configuration issues from upstream behavior. Done means each symptom is explained and the appropriate configuration or upstream fix is identified with regression coverage where applicable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100