argoproj / argoproj/argo-cd

Application sync stuck when using PVC with WaitForFirstConsumer storage binding mode

Open
#12,840 4 comments 6 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
24.2k
Forks
7.9k
Avg merge
1d 6h
Merged PRs (30d)
239

Description

Checklist:

* [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
* [x] I've included steps to reproduce the bug.
* [x] I've pasted the output of `argocd version`.

**Describe the bug**

When deploying an app that contains a PVC to a storage class with `volumeBindingMode` set to `WaitForFirstConsumer`, the application sync is stuck due to a deadlock: the pod won't be created by Argo until the PVC is ready, and the PVC waits for a pod to use the claim before being ready.

See https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode

**To Reproduce**

StorageClass
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gp3-csi
provisioner: ebs.csi.aws.com
parameters:
encrypted: 'true'
type: gp3
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
```

**Expected behavior**

ArgoCD should handle this deadlock by continuing with pod creation when the PVC is in state `WaitForFirstConsumer`

**Screenshots**

n/a

**Version**

```shell
{
"Version": "v2.6.1+3f143c9",
"BuildDate": "2023-02-08T18:51:05Z",
"GitCommit": "3f143c9307f99a61bf7049a2b1c7194699a7c21b",
"GitTreeState": "clean",
"GoVersion": "go1.18.10",
"Compiler": "gc",
"Platform": "linux/amd64",
"KustomizeVersion": "v4.5.7 2022-08-02T16:35:54Z",
"HelmVersion": "v3.10.3+g835b733",
"KubectlVersion": "v0.24.2",
"JsonnetVersion": "v0.19.1"
}
```

**Logs**

![image](https://user-images.githubusercontent.com/25197472/224771315-f64e161c-3971-4961-b435-0b95a7ba0a41.png)

Contributor guide

Open the contributing guide

Research direction

Start with the provided StorageClass and PVC reproduction, then trace Argo CD's sync and resource-health handling for PVCs and pods. Confirm the behavior against Kubernetes' WaitForFirstConsumer semantics; done means an application containing the PVC progresses to pod creation instead of remaining stuck.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.