apache / apache/cloudstack

CKS: make the source CIDR of the auto-provisioned node SSH firewall rules configurable

Ouverte
#13,970 1 commentaire 2 réactions 0 personnes assignées Voir sur GitHub
component:cks component:networking
Langage dominant
Java
Étoiles
3.1k
Forks
1.4k
Merge moyen
6 j 19 h
PR mergées (30 j)
32

Description

As an Operator I would like to be able to restrict the source CIDR of the firewall rules that
CKS provisions for node SSH access, instead of having them permanently opened to `0.0.0.0/0`.

### Current behaviour

When a Kubernetes cluster is created on an isolated network, CKS provisions ingress firewall
rules on the network's source NAT IP for the node SSH ports (`2222` .. `2222 + nodes - 1`,
plus one extra rule per external node). The source CIDR is hard-coded:

```java
// plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/
// actionworkers/KubernetesClusterActionWorker.java
protected void provisionFirewallRules(final IpAddress publicIp, final Account account,
int startPort, int endPort) throws ... {
List sourceCidrList = new ArrayList();
sourceCidrList.add("0.0.0.0/0"); // <-- hard-coded
...
cidrField.set(rule, sourceCidrList);
firewallService.createIngressFirewallRule(rule);
firewallService.applyIngressFwRules(publicIp.getId(), account);
}
```

It is reached from `addFirewallRulesForNodes()`. There is no way to override the value:
`KubernetesClusterManagerImpl.getConfigKeys()` exposes 15 keys (timeouts, network offering,
max cluster size, etcd start port, ...) and none of them relates to network access, and
`CreateKubernetesClusterCmd` has no corresponding parameter.

The net effect is that on every CKS cluster deployed on an isolated network, SSH of every
control and worker node is reachable from the whole internet, and the operator has no
supported way to change that.

### Why narrowing the rule by hand is not a workaround

Editing the rule does not survive normal lifecycle operations.
`KubernetesClusterScaleWorker.scaleKubernetesClusterIsolatedNetworkRules()` calls
`removeSshFirewallRule()`, which matches the rule **by port number only**:

```java
if (Objects.equals(firewallRule.getSourcePortStart(), CLUSTER_NODES_DEFAULT_START_SSH_PORT)
|| (Objects.nonNull(pfRule) && pfRule.getDestinationPortStart() == DEFAULT_SSH_PORT)) {
rule = firewallRule;
firewallService.revokeIngressFwRule(firewallRule.getId(), true);
break;
}
```

The narrowed rule is therefore revoked and then recreated with `0.0.0.0/0` by
`setupKubernetesClusterIsolatedNetworkRules()`. The same happens on
`addNodesToKubernetesCluster`. Because the public ports of the SSH port-forwarding rules are
fixed by CKS itself, there is no way to express a narrowed rule that this matcher would not
pick up. `cidrlist` of an existing firewall rule is immutable, so it cannot be edited in place
either.

### Prior discussion

This has been raised before, inside bug reports about something else:

* [#11779](https://github.com/apache/cloudstack/issues/11779) describes exactly this scenario —
the reporter removed the default wide-open rules for security reasons, after which cluster
scaling failed with `ManagementServerException: Firewall rule for node SSH access can't be
provisioned`. Closed as a duplicate of #11758.
* In [#11758](https://github.com/apache/cloudstack/issues/11758), @weizhouapache confirmed
(2025-10-01) that *"several code lines are based on the assumption that the firewall rules
and port forwarding rules for SSH (to control/worker nodes) start from port 2222"*, and when
asked specifically about the security risk of opening 6443 and 2222–22xx to `0.0.0.0/0`,
answered (2025-10-03): *"I understand your concerns. I agree we should improve it. it is not
a simple fix, please keep eye on this issue"*.

[#12806](https://github.com/apache/cloudstack/pull/12806) then closed #11758. That PR fixed the
robustness side of the problem — a missing or NULL-ported rule no longer throws — but
intentionally left the hard-coded CIDR untouched. As a result the security aspect is currently
not tracked by any open issue, which is why I am opening this one.

### Proposed feature

Add a configuration key, for example `cloud.kubernetes.cluster.ssh.allowed.cidr`, scoped to
Account or Domain, defaulting to `0.0.0.0/0` so that existing behaviour is preserved, and use
its value in `provisionFirewallRules()` in place of the constant. Optionally expose the same
value as a parameter of `createKubernetesCluster` so it can be set per cluster.

Two related points worth covering by the same setting:

* The API-port rule (6443) provisioned for the load balancer / port forwarding has the same
issue.
* For VPC-based clusters the equivalent path is `createVpcTierAclRules()` →
`provisionVpcTierAllowPortACLRule()`, which builds a `CreateNetworkACLCmd` without setting
`cidrlist`, so `getSourceCidrList()` falls back to `0.0.0.0/0` and `::/0`.

A more thorough alternative would be to stop relying on the public IP for management-server →
node SSH altogether (the shared-network path already uses the node's private address via
`getKubernetesClusterServerIpSshPortForSharedNetwork()`), but a configurable CIDR would already
remove the immediate exposure with a very small change.

### Versions

Observed on 4.22.1.0; the code paths above are unchanged in `main`.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez dans plugins/integrations/kubernetes-service/src/main/java/com/cloud/kubernetes/cluster/actionworkers/KubernetesClusterActionWorker.java, en suivant addFirewallRulesForNodes() jusqu’à provisionFirewallRules(), puis examinez KubernetesClusterManagerImpl.getConfigKeys() et CreateKubernetesClusterCmd. Comparez les chemins de réseau isolé avec createVpcTierAclRules() et provisionVpcTierAllowPortACLRule(). Le travail est terminé lorsque le CIDR configuré est appliqué aux règles SSH et API/ACL pertinentes, tout en préservant le comportement par défaut.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
java, kubernetes
Domaine
cloud, infrastructure, security
Type d'issue
Fonctionnalité
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
52/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.