validateExclusiveAuthSettings treats Arc and bootstrap-token as mutually exclusive
- Dominant language
- Go
- Stars
- 14
- Forks
- 28
- Avg merge
- 4d 13h
- Merged PRs (30d)
- 12
Description
Hi team, second of three related observations on the Arc-mode kubelet auth path. Cross-links to the others at the bottom.
## What we observed
`validateExclusiveAuthSettings` in `pkg/config/validate.go` treats `arc.enabled` and `bootstrapToken` as mutually exclusive. Picking either appears to disable the other across the whole agent.
In our scenario these two felt like they were doing different jobs:
- `arc.enabled` controls the agent's own Azure machine identity (used for `enrich-cluster-config`, `assignRBACRoles`, AKS RP calls, etc).
- `bootstrapToken` controls how kubelet authenticates to the API server.
When we picked `bootstrapToken` to work around the MI exec issue (see related issue below), Arc was also disabled, which disabled `enrich-cluster-config`. The agent then no longer refreshed the cluster CA on its own, and when the API-server CA rotated (this happened after we enabled AAD on the cluster), the bootstrap kubeconfig went stale and kubelet bootstrap failed until we manually re-fetched the CA.
## Impact for our scenario
The Arc-mode MI happy path doesn't yet work end-to-end for us (separate issue), so the only path that actually completes is "bootstrap-token for kubelet". But that path also disables Arc for the agent, which gives up `enrich-cluster-config` and leaves us with a stale CA on rotation. Feels like we're being forced to pick one cost or the other.
## One possible direction
Would it be reasonable to treat these two as orthogonal? Roughly:
| `arc.enabled` | `bootstrapToken` | `node.kubelet.auth.mode` | Resulting behavior |
|---|---|---|---|
| true | unset | mi | Arc-driven everything; kubelet uses MI exec |
| true | set | bootstraptoken | Arc for the agent's Azure-side ops (incl. `enrich-cluster-config`), bootstrap-token brings kubelet up |
| false | set | bootstraptoken | "Bring-your-own-everything" path (existing VM demos) |
The middle row would unblock us today without depending on the MI exec fix landing first. Totally happy to be told we're holding it wrong if there's a reason these are exclusive that we missed.
Version tested: v0.1.0 (git 65d8d38).
---
## Related on this auth path (tightly coupled)
- #185 — Kubelet MI exec credential does not propagate Arc HIMDS env vars
- #187 — Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available
---
## Other observations from the same Pi-rack edge demo
- #182 — Arc machine RG and AKS cluster RG must match (hard-coded in v0.1.0)
- #183 — Agent should preflight RBAC permissions before `assignRBACRoles`
- #184 — Hostname casing not normalized; PascalCase machine names fail node validation
- #185 — Kubelet MI exec credential does not propagate Arc HIMDS env vars
- #187 — Agent could auto-mint kubelet bootstrap-token when Arc/MI/SP credentials are available
- #188 — `kube1` systemd-nspawn launcher passes `--network-veth` but actually shares host netns
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in pkg/config/validate.go at validateExclusiveAuthSettings and trace how arc.enabled, bootstrapToken, and node.kubelet.auth.mode are validated and applied. Compare the current behavior with the three configuration combinations in the issue, then verify that Arc agent operations and bootstrap-token kubelet authentication can coexist without disabling either path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- authentication
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100