fosrl / fosrl/helm-charts

controller mode crash-loops out of the box: generated NetworkPolicy blocks controller egress to the Kubernetes API

Open
#22 2 comments 0 reactions 1 assignee Claimed by @marcschaeferger View on GitHub
bug
Dominant language
Shell
Stars
17
Forks
11
Avg merge
2d 6h
Merged PRs (30d)
8

Description

**Chart:** pangolin 0.1.0-alpha.1 (repo https://charts.fossorial.io)
**Values:** deployment.type: controller, deployment.mode: multi, networkPolicy.enabled: true (default)
**Environment:** k3s v1.36 (single node), Traefik installed separately via Helm, NetworkPolicy enforcement via k3s's default controller

**Symptom**

With defaults, pangolin-kube-controller crash-loops immediately:

level=info msg="Checking for required Traefik CRDs..."
level=error msg="application error: CRD validation failed: required CRD \"middlewares.traefik.io\" not found:
Get \"https://10.43.0.1:443/apis/apiextensions.k8s.io/v1/customresourcedefinitions/middlewares.traefik.io\":
dial tcp 10.43.0.1:443: connect: connection refused"

The CRDs exist; the controller simply cannot reach the Kubernetes API.

**Cause**

The generated *-controller-egress NetworkPolicy allows only DNS (53 to kube-system) and TCP 3001 to the Pangolin pod. The API egress rule (networkPolicy.controller.egress.kubernetesApi) is skipped because cidr defaults to "" — so the chart's own recommended mode ships with its controller unable to perform its core function.

Two additional limitations even when cidr is set:
1. It's a single CIDR + single port, but clusters commonly need two shapes: the API service IP on 443 (CNIs enforcing pre-DNAT) and the node/endpoint IP on 6443 (CNIs enforcing post-DNAT, e.g. k3s). The legacy kubernetesApiCIDRs list allows multiple CIDRs but still one port.
2. Neither value is discoverable from the failure mode — "connection refused to 10.43.0.1" doesn't point at a NetworkPolicy.

**Suggested fix**

Default the controller policy to allow TCP 443 + 6443 egress (optionally to 0.0.0.0/0 when no CIDR is provided), or support a list of {cidr, port} pairs. At minimum, document that controller mode requires setting networkPolicy.controller.egress.kubernetesApi.cidr and fail loud in the chart's NOTES.

**Workaround**

A supplementary NetworkPolicy selecting the controller pods with egress on TCP 443/6443 restores function without modifying the chart.

**Related:** the ingress policy defaults also block the dashboard port (3002) unless networkPolicy.pangolin.externalingressRoute.dashboard.enabled:true defaulting the dashboard on, the out-of-box combination 502s. Possibly intended, but worth aligning defaults.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.