kubernetes-sigs / kubernetes-sigs/cluster-api

Improve cluster-autoscaler integration

Open
#12,762 14 comments 0 reactions 1 assignee Claimed by @sivchari View on GitHub
area/autoscaling help wanted kind/feature priority/important-soon triage/accepted
Dominant language
Go
Stars
4.3k
Forks
1.6k
Avg merge
1d 3h
Merged PRs (30d)
113

Description

While investigating an [autoscaler issue](https://github.com/kubernetes/autoscaler/issues/8494) we identified a few areas of improvement of the autoscaler - CAPI integration

We should look into the following areas:
* [x] First temporary/stopgap solution for CAPI controller and autoscaler fighting over replicas during MD rollouts (see: https://github.com/kubernetes/autoscaler/issues/8494)
* [ ] Improve behavior during Machine deletion (including Node drain etc.)
* Today autoscaler cordons/taints/drains Nodes before triggering Machine deletion
* This means that the CAPI Machine deletion logic is not respected (pre-drain hooks, MachineDrainRules, drain observability, ...)
* An idea: Maybe we want to disable cordon/taint/drain in autoscaler, options:
* Via a global flag to allow disabling drain
* Extend the `CloudProvider` interface with a new Method to allow disabling drain per node group
* Extending the `GetOptions` method of the `NodeGroup` interface to allow disabling drain per node group
* [ ] Double-check that autoscaler does not scale up to many Machines based on pending Pods
* Not entirely sure, but it looks like we observed autoscaler scaling up twice within 12 seconds because of just 1 pending Pod
* [ ] Find a final solution for autoscaling during MD rollouts
* [ ] Improve how autoscaler triggers Machine deletion (`delete-machine` annotation on MS-level + MD scale down is a weak/no API)
* [ ] Specifically consider auto-scaling behavior during in-place updates

Note: The following seems to be available today:
* Disable cordon with: `--cordon-node-before-terminating=false` (but disabling cordon without disabling drain seems bad)
* Disable `DeletionCandidateOfClusterAutoscaler:PreferNoSchedule` taint with: `--max-bulk-soft-taint-count=0`
* `ToBeDeletedByClusterAutoscaler:NoSchedule` taint cannot be disabled today, needs a new flag (like one of the two above) (Hack: overwrite `maxConcurrentNodesTainting` const to 0?)

Past Slack thread: https://kubernetes.slack.com/archives/C8TSNPY4T/p1756465597770899

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.