bpfman / bpfman/bpfman-operator

Config: allow tolerations for the bpfman daemon DaemonSets

Open
#531 0 comments 8 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
38
Forks
27
PR merge metrics
No merged PRs in 30d

Description

### What would you like to be added?

An optional `tolerations` field on the `Config` CRD, so the bpfman daemon and metrics-proxy DaemonSets can schedule onto nodes with custom taints without resorting to the `spec.overrides` escape hatch.

```yaml
apiVersion: bpfman.io/v1alpha1
kind: Config
metadata:
name: bpfman-config
spec:
namespace: bpfman
tolerations:
- key: workload
operator: Equal
value: gpu
effect: NoSchedule
```

### Why is this needed?

The daemon DaemonSet is rendered from a static manifest (`config/bpfman-deployment/daemonset.yaml`) whose tolerations are hardcoded to the control-plane/master taints. On clusters with custom taints — infrastructure pools, GPU pools, dedicated workloads — the daemon cannot reach those nodes at all.

That is not a partial degradation: the operator expects a `BpfApplicationState` object from every node, so a single uncovered node leaves **every** `BpfApplication` in the cluster stuck in `Pending`.

This was previously achievable when the daemon was deployed via a ConfigMap containing the raw DaemonSet YAML. With the move to the `Config` CRD the only remaining option is marking the DaemonSet `unmanaged` via `spec.overrides`, which disables all reconciliation for it.

### Proposed behavior

- `tolerations` lives on `ConfigSpec` rather than `DaemonSpec`, since it must reach the metrics-proxy DaemonSet too.
- Configured tolerations are **merged** into both DaemonSets on top of the static manifest's, skipping duplicates. Replacing would drop the built-in control-plane/master tolerations and hit the `Pending` problem described above.

### Note

This issue originally also covered `nodeSelector` and `affinity`. Restricting the daemon to a subset of nodes hits the same node-coverage invariant and needs a dedicated effort — that is now tracked in #542.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with ConfigSpec and the Config reconciliation path, then inspect config/bpfman-deployment/daemonset.yaml and the metrics-proxy DaemonSet rendering. Trace how static tolerations are copied into both DaemonSets and identify where Config values can be merged without removing the built-in entries. Done means the CRD accepts tolerations, duplicates are skipped, and both DaemonSets receive the configured values while retaining their defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.