bpfman / bpfman/bpfman-operator
Config: allow tolerations for the bpfman daemon DaemonSets
- Dominant language
- Go
- Stars
- 38
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
### What would you like to be added?
An optional `tolerations` field on the `Config` CRD, so the bpfman daemon and metrics-proxy DaemonSets can schedule onto nodes with custom taints without resorting to the `spec.overrides` escape hatch.
```yaml
apiVersion: bpfman.io/v1alpha1
kind: Config
metadata:
name: bpfman-config
spec:
namespace: bpfman
tolerations:
- key: workload
operator: Equal
value: gpu
effect: NoSchedule
```
### Why is this needed?
The daemon DaemonSet is rendered from a static manifest (`config/bpfman-deployment/daemonset.yaml`) whose tolerations are hardcoded to the control-plane/master taints. On clusters with custom taints — infrastructure pools, GPU pools, dedicated workloads — the daemon cannot reach those nodes at all.
That is not a partial degradation: the operator expects a `BpfApplicationState` object from every node, so a single uncovered node leaves **every** `BpfApplication` in the cluster stuck in `Pending`.
This was previously achievable when the daemon was deployed via a ConfigMap containing the raw DaemonSet YAML. With the move to the `Config` CRD the only remaining option is marking the DaemonSet `unmanaged` via `spec.overrides`, which disables all reconciliation for it.
### Proposed behavior
- `tolerations` lives on `ConfigSpec` rather than `DaemonSpec`, since it must reach the metrics-proxy DaemonSet too.
- Configured tolerations are **merged** into both DaemonSets on top of the static manifest's, skipping duplicates. Replacing would drop the built-in control-plane/master tolerations and hit the `Pending` problem described above.
### Note
This issue originally also covered `nodeSelector` and `affinity`. Restricting the daemon to a subset of nodes hits the same node-coverage invariant and needs a dedicated effort — that is now tracked in #542.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ConfigSpec and the Config reconciliation path, then inspect config/bpfman-deployment/daemonset.yaml and the metrics-proxy DaemonSet rendering. Trace how static tolerations are copied into both DaemonSets and identify where Config values can be merged without removing the built-in entries. Done means the CRD accepts tolerations, duplicates are skipped, and both DaemonSets receive the configured values while retaining their defaults.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100