feat(planner): detect sidecar.image drift on Running nodes
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
Research direction
Start with internal/planner/planner.go:651-658 and compare buildNodeUpdatePlan at :686-694, then read internal/planner/node_update_test.go and internal/planner/doc.go:37. Trace how the latest pod or StatefulSet image is observed before choosing the sidecar drift path. Done means a Running SeiNode sidecar-only image change builds and completes a plan, status reflects the running image, the analogous test passes, and the doc comment covers sidecar drift.
Written by the indexing model from the issue text.
Description
Problem
buildRunningPlan in internal/planner/planner.go:651-658 detects two kinds of drift on a Running SeiNode and builds nothing else:
func buildRunningPlan(node *seiv1alpha1.SeiNode) (*seiv1alpha1.TaskPlan, error) {
if node.Spec.Image != node.Status.CurrentImage {
return buildNodeUpdatePlan(node) // main seid image drift
}
if sidecarNeedsReapproval(node) {
return buildMarkReadyPlan(node) // sidecar Ready=False condition
}
return nil, nil
}
There is no third check that compares spec.template.spec.sidecar.image against the live StatefulSet's sidecar container image. So if an operator updates only the sidecar image:
- The CRD spec updates.
buildRunningPlansees no drift (main image unchanged, sidecar Ready condition unchanged).- Returns nil — no plan, no rollout.
- StatefulSet on disk still has the old sidecar image.
- Pods keep running the old sidecar until something else triggers a restart (operator's
kubectl delete pod, a main-image bump that incidentally re-applies, or a full SND disable+re-enable).
Concrete impact: rolling out a sidecar bug fix (e.g., the seictl#123 snapshot-restore regex fix shipped in image 271ab686…) currently requires either piggybacking on a main image bump or a heavy SND re-creation. Operators wanting to upgrade only the sidecar today have no clean path.
Repro
- Create or take an existing Running
SeiNodewithspec.template.spec.sidecar.image: imageA. - Edit the spec to
imageB(different sidecar image, no main image change). - Reconcile: no plan is built. The pod continues running
imageAindefinitely.
Two design options
A. SidecarUpdate plan as a third drift check
Mirror the NodeUpdate shape:
func buildRunningPlan(node *seiv1alpha1.SeiNode) (*seiv1alpha1.TaskPlan, error) {
if node.Spec.Image != node.Status.CurrentImage {
return buildNodeUpdatePlan(node)
}
if sidecarImageDrift(node) { // NEW
return buildSidecarUpdatePlan(node)
}
if sidecarNeedsReapproval(node) {
return buildMarkReadyPlan(node)
}
return nil, nil
}
Where sidecarImageDrift compares spec.template.spec.sidecar.image (or default if unset) to the observed sidecar image on the most recent pod / StatefulSet. Tracking the observed image probably wants a new status.currentSidecarImage field for symmetry with currentImage.
buildSidecarUpdatePlan task sequence: apply-statefulset → apply-service → observe-sidecar-image → mark-ready. (The observe-sidecar-image task is a small new task that polls the StatefulSet rollout and stamps currentSidecarImage.)
Pros: explicit, mirrors existing pattern, easy to reason about.
Cons: a second drift-detection field to maintain, a new task type.
B. Always reconcile the StatefulSet on every Running reconcile
Replace plan-driven apply-statefulset for steady-state with an idempotent server-side-apply on every reconcile. The kubelet handles pod rotation when the sidecar image actually changes (StatefulSet rollout policy). The drift detection becomes implicit: SSA is a no-op when nothing changed.
Pros: simpler conceptually, removes an entire class of "did we forget a drift check" bugs (any field — not just sidecar.image — would propagate).
Cons: changes the contract that "the executor only mutates owned resources during plan tasks." Less obvious where rotates come from when reading logs. Need to confirm SSA idempotency interactions with the per-task field manager (seinode-controller).
Recommendation
Option A first — smaller scope, closer to the existing pattern, easier to write a focused test for. Option B is a refactor worth its own design pass.
Acceptance criteria
-
spec.template.spec.sidecar.imagechange on a RunningSeiNodetriggers a plan that ends with the pod actually running the new sidecar image, without needing a main-image bump or SND re-creation. -
status.currentSidecarImage(or equivalent) reflects the running sidecar. - Test analogous to
internal/planner/node_update_test.gocovers the new path. - Doc comment in
internal/planner/doc.gomentions sidecar drift alongside main-image drift.
References
internal/planner/planner.go:651-658(steady-state drift logic)internal/planner/planner.go:686-694(existingbuildNodeUpdatePlanto mirror)internal/planner/doc.go:37(drift detection docstring)internal/planner/node_update_test.go(test pattern to mirror)- Surfaced during platform fork-test work where seictl#123 needed to roll out as a sidecar-only update.
- Dominant language
- Go
- Stars
- 1
- Forks
- 2
- Avg merge
- 2h 29m
- Merged PRs (30d)
- 56
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from sei-protocol/sei-k8s-controller
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
-
Difficulty 5/5 Over a week Newbie friendliness 32/100
sei-protocol/sei-k8s-controller#457 · 2 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 4/5 3-5 days Newbie friendliness 45/100
-
Difficulty 3/5 1-2 days Newbie friendliness 65/100
All issues in sei-protocol/sei-k8s-controller
Similar issues
-
optimization optimization:agents-md-curator
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
githubnext/gh-aw-cao#13143 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
blinklabs-io/bursa#904 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
yanet-platform/ipfw-go#129 ·
-
bug confmap/provider/googlesecretmanagerprovider needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
open-telemetry/opentelemetry-collector-contrib#51273 · 2 comments ·
-
bug: AI Gateway client filter lists "Unknown" twice when NULL and literal Unknown clients coexist Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100