kubernetes-sigs / kubernetes-sigs/cluster-api
Support node draining before starting in-place update
- Dominant language
- Go
- Stars
- 4.3k
- Forks
- 1.6k
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 113
Description
### What would you like to be added (User Story)?
As in-place update runtime hook developer I'd like to notify CAPI that node draining and/or volume detaching is required in order to prepare nodes for a disruptive update
### Detailed Description
**Context**
We manage thousands of bare metal servers as k8s nodes using Cluster API. During immutable rollout we have to wipe the server and then provision probably the same server. Due to the internal implementation, the "wipe" phase takes an additional 10 minutes while technically we could just "re-setup" the same server. So, we'd like to address this issue by using the in-place update feature in order to skip the delete phase and run re-setup during update.
**Feature request**
In order to safely update node we have to perform draining and volume detaching just like the Machine controller does, so we'd really like to reuse this logic. I see a few options for how it could be implemented:
* Return additional "requirements" (drain / node volume detach) from the CanUpdateMachineSet hook (CanUpdateMachine probably too, but we're interested in worker update first)
* MD controller will be able to select the safest extension: prioritize ones that don't require drain over ones that do
* The requirements can be propagated to the Machine controller via annotations
* Machine controller will reuse the same drain and volume detach logic before the update starts, and uncordon the node when update is finished and readiness gates are passed
* Encapsulate the drain and volume detach logic into an exported package and prepare something like middleware to make it usable in hooks
**Why not to do X**
* Implement it ourselves -- we do want draining and volume detaching to be consistent between delete and update scenario
### Anything else you would like to add?
The described need is specific, but I believe that much more "in-place" (without machine recreate) update scenarios exist, that will affect workloads on the node.
### Label(s) to be applied
/kind feature
/area machine
/area machineset
/area machinedeployment
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.
Contributor guide
Research direction
Start by reading the CanUpdateMachineSet and CanUpdateMachine hook proposals, then compare them with the Machine controller's existing drain and volume-detach behavior. Evaluate how requirements could be propagated and when nodes would be uncordoned; done means selecting and documenting an implementation approach that safely reuses this lifecycle behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- backend, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100