kubermatic / kubermatic/operating-system-manager

[bug] cloud-init configuration: cloud config \"bootstrap\" is not ready yet

Open
#567 3 comments 2 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
53
Forks
43
Avg merge
1d 2h
Merged PRs (30d)
15

Description

Hi,

We have multiple observations of this error across multiple customers that only "sometimes" the user-cluster MD does not create new Nodes and the logs throw below error in machine-controller pod.

```
{"level":"error","time":"2026-02-27T06:03:38.291Z","logger":"machine-controller","caller":"machine/controller.go:388","msg":"Reconciling failed","machine":"kube-system/xxxxx-nngcb","error":"cloud-init configuration: cloud config \"bootstrap\" is not ready yet"}
```

The error - upon investigation happens due to failure from OSM controller to update the bootstrap secret. Restarting machine-controller pods or OSM pods does not make desired impact. Only way to resolve the situation is to manually delete the `-kube-system-bootstrap-config` secret in user-cluster's `cloud-init-settings` namespace. After deletion of the secret, if we restart OSM pod, it recreates those secrets properly and then machine-controller can create the nodes.

We don't have a reproducible conditions for this. But mostly it happens when there is a long time between OSP change and node rotation like a month etc.

We have seen this at two customers between Myself and Akash. Also it seems Demo env also faces this issue. So My request is to "generally" look at the code to see why will OSC ignores to update the secret even when it should.

Contributor guide

Open the contributing guide

Research direction

Start at machine/controller.go:388 and trace how the OSM controller updates the machine-deployment-kube-system-bootstrap-config secret in the cloud-init-settings namespace. Investigate why the bootstrap secret can remain unready after a long interval between OSP changes and node rotation; done means the secret is updated or recreated without manual deletion and machine-controller can create the nodes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.