volcano-sh / volcano-sh/agentcube

mtls: Pods restart multiple times when SPIRE is first enabled

Open
#374 2 comments 0 reactions 1 assignee View on GitHub

@avinxshKD is already working on this.

Since Jul 6, 2026.

kind/bug
Dominant language
Go
Stars
167
Forks
88
Avg merge
44m
Merged PRs (30d)
1

Description

What happened:

When SPIRE is enabled via helm upgrade --set spire.enabled=true, the Router and WorkloadManager pods crash and restart several times before stabilizing (Router: 5 restarts, WM: 3, SPIRE Agent: 3).

The root cause is a startup race condition. The spiffe-helper sidecar and the main container start simultaneously, but the main container needs cert files that spiffe-helper hasn't written yet. Both Router (cmd/router/main.go:66) and WorkloadManager (cmd/workload-manager/main.go:82) call klog.Fatalf if certs aren't found within 30s (pkg/mtls/wait.go:30), crashing the pod.

What you expected to happen:
Pods should start cleanly with 0 restarts.

How to reproduce it (as minimally and precisely as possible):

  1. Install AgentCube without SPIRE on a Kind cluster.
  2. Run helm upgrade with --set spire.enabled=true.
  3. Watch pods via kubectl get pods -n agentcube -w. observe multiple CrashLoopBackOff cycles.

Anything else we need to know?:

Suggested fix: Convert spiffe-helper from a regular sidecar to a Kubernetes native sidecar by moving it to initContainers with restartPolicy: Always . This guarantees Kubernetes starts spiffe-helper and lets it write the cert files before the main container starts.

Environment:

  • agentcube version:
  • Kubernetes version: v1.32.2 (Kind v0.27.0)
  • Others: Helm v3.17.3

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.