openshift / openshift/cluster-kube-controller-manager-operator

kcm rootCA missing apiserver ca leads to kube-root-ca problem

Open
#702 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

lifecycle/frozen
Dominant language
Go
Stars
45
Forks
152
Avg merge
17h 32m
Merged PRs (30d)
6

Description

1 Bug phenomenon

  1. The kcm(kube-controller-manager) rootCA is generated by manageServiceAccountCABundle in targerconfigcontroller, and this func will get kube-apiserver-server-ca cm first , and then use it and other two cm to generate kcm rootCA
  2. when sometimes(very unlikely to happen , but i have met this just one time) the kube-apiserver-server-ca cm is missing, and the manageServiceAccountCABundle generate rootCA without kube-apiserver-server-ca , and finally the kcm leader holds the wrong rootCA, it will lead to the kube-root-ca problem in every pod, and the ocp release wil not work until i stop the wrong kcm leader

2 Bug fix:
2.1. this bug can be resolved by adding kube-apiserver-server-ca check in manageServiceAccountCABundle of targerconfigcontroller ,as follows
https://github.com/openshift/cluster-kube-controller-manager-operator/blob/4ca346ef97def3697f1aa0368c9b35459b9b2f59/pkg/operator/targetconfigcontroller/targetconfigcontroller.go#L706-L720
image
2.1 maybe like this , but this is not the best way to resolve this problem
image

2.3 this is also can be resolved by modifying the openshift library func CombineCABundleConfigMaps in resourcesynccontroller as
https://github.com/openshift/cluster-kube-controller-manager-operator/blob/4ca346ef97def3697f1aa0368c9b35459b9b2f59/vendor/github.com/openshift/library-go/pkg/operator/resourcesynccontroller/core.go#L17-L67
image

3 This is related to the openshift library-go issue # issue 1472 github.com/openshift/library-go missing key configmap

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/operator/targetconfigcontroller/targetconfigcontroller.go at manageServiceAccountCABundle, then compare the behavior with vendor/github.com/openshift/library-go/pkg/operator/resourcesynccontroller/core.go and library-go issue #1472. Verify how a missing kube-apiserver-server-ca ConfigMap affects the generated root CA and inspect relevant existing controller tests. Done means the missing input cannot produce an incorrect root CA and the failure behavior is covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.