argoproj / argoproj/argo-cd

ArgoCD Application Stuck in OutOfSync When Deploying Multiple CRDs

Open
#21,079 9 comments 4 reactions 0 assignees View on GitHub
bug component:sync feature:server-side-apply version:2.13
Dominant language
Go
Stars
24.2k
Forks
7.9k
Avg merge
1d 6h
Merged PRs (30d)
239

Description

Checklist:

* [x] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
* [x] I've included steps to reproduce the bug.
* [x] I've pasted the output of `argocd version`.

**Describe the bug**

When an `Application` includes multiple CRDs to deploy (in my case, 3 or more), the Application gets stuck in the `OutOfSync` state, even though the CRDs are deployed successfully.
It seems like ArgoCD doesn't recognize that these resources have been deployed correctly until you manually refresh/sync.
In our case, we use push-based deployment, so we have to synchronize it manually. Otherwise, the issue resolves itself after 3 minutes when ArgoCD performs another sync.

**To Reproduce**

Create a local kind cluster
```bash
kind create cluster --name "argocd-demo" --image "kindest/node:v1.29.8"
```

Install ArgoCD with default values
```bash
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm upgrade --install argocd argo/argo-cd \
--version "7.7.2" \
--namespace argocd \
--create-namespace \
--set configs.params."server\.insecure"=true \
--set global.logging.level="debug" \
--set global.image.tag="v2.13.0"
```

Deploy this application. This application deploy six cert-managers CRDs (cert-manager is not the problem, we have the same problem with linkerd). This application has been simplified as much as posible.
```yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: 'cert-manager-crds'
namespace: argocd
spec:
project: default
source:
repoURL: https://github.com/hlastras/test-argocd-crds.git
targetRevision: HEAD
path: services/cert-manager/multi-file # same problem if I add all crds on a single file
destination:
server: https://kubernetes.default.svc
syncPolicy:
automated: {}
syncOptions:
- ServerSideApply=true
```

**Expected behavior**

If all the CRDs are deployed successfully, the `Application` should be flagged as `Healthy` and `Synced`.

**Screenshots**

Screenshot 2024-12-05 at 18 52 21
Screenshot 2024-12-05 at 18 52 37

BUT

Screenshot 2024-12-05 at 18 53 11

**Version**

I've tested this with ArgoCD versions `1.13.0` and `1.12.1`, both installed using Helm.
When I installed it using the command:
`kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/v2.13.0/manifests/install.yaml`
the behavior remained the same (application stuck on OutOfSync). However, in this case in addition I received a warning about a resource on two applications.

**Logs**

application-controller logs in debug mode (deleted extra-large logs that contain the full CRD text):
```
2024/12/05 17:50:31 maxprocs: Leaving GOMAXPROCS=6: CPU quota undefined
time="2024-12-05T17:50:31Z" level=info msg="ArgoCD Application Controller is starting" built="2024-11-04T12:09:06Z" commit=347f221adba5599ef4d5f12ee572b2c17d01db4d namespace=argocd version=v2.13.0+347f221
time="2024-12-05T17:50:31Z" level=info msg="Processing all cluster shards"
time="2024-12-05T17:50:31Z" level=debug msg="Processing clusters from shard 0: Using filter function: legacy"
time="2024-12-05T17:50:31Z" level=info msg="Processing all cluster shards"
time="2024-12-05T17:50:31Z" level=info msg="appResyncPeriod=3m0s, appHardResyncPeriod=0s, appResyncJitter=0s"
time="2024-12-05T17:50:31Z" level=info msg="Starting configmap/secret informers"
time="2024-12-05T17:50:31Z" level=info msg="Configmap/secret informer synced"
time="2024-12-05T17:50:31Z" level=warning msg="The cluster https://kubernetes.default.svc has no assigned shard."
time="2024-12-05T17:50:31Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:50:31Z" level=info msg="Cluster https://kubernetes.default.svc has been assigned to shard 0"
time="2024-12-05T17:50:31Z" level=info msg="Start watch clusters"
time="2024-12-05T17:50:31Z" level=debug msg="Skipping sharding distribution update. Cluster already added"
time="2024-12-05T17:50:31Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:50:31Z" level=debug msg="Checking if cache &{0x4001184660 0x4000226790 0x2246db0 0x4001184420 0x4000708480 0x40003c3570 0x40007f1c20 0x7d2a060 {0} map] {{map[apiextensions.k8s.io/CustomResourceDefinition:{ false {[/status /spec/preserveUnknownFields] ] ]} {] ] ]} ]}] 0x40011d2900} argocd.argoproj.io/instance label map[*/*:{ false {[/metadata/resourceVersion /metadata/generation /metadata/managedFields] ] ]} {] ] ]} ]} apiextensions.k8s.io/CustomResourceDefinition:{ false {] ] ]} {] ] ]} ]}] false} {{0 0} 0 0 {{} 0} {{} 0}}} / cluster &Cluster{Server:https://kubernetes.default.svc,Name:in-cluster,Config:ClusterConfig{Username:,Password:,BearerToken:,TLSClientConfig:TLSClientConfig{Insecure:false,ServerName:,CertData:nil,KeyData:nil,CAData:nil,},AWSAuthConfig:nil,ExecProviderConfig:nil,},ConnectionState:ConnectionState{Status:Successful,Message:,ModifiedAt:2024-12-05 17:50:31.488342159 +0000 UTC m=+0.165670061,},ServerVersion:1.29,Namespaces:],RefreshRequestedAt:,Info:ClusterInfo{ConnectionState:ConnectionState{Status:,Message:,ModifiedAt:,},ServerVersion:,CacheInfo:ClusterCacheInfo{ResourcesCount:0,APIsCount:0,LastCacheSyncTime:,},ApplicationsCount:0,APIVersions:],},Shard:nil,ClusterResources:false,Project:,Labels:map[string]string{},Annotations:map[string]string{},} has appInformer &{0x40007ee0f0 0x40013180b0 0x40007f1e00 {} 0x40007ee0d8 0x4000d22c08 180000000000 180000000000 0x7d2a060 true false {0 0} {0 0} }"
time="2024-12-05T17:50:31Z" level=info msg="Starting secretInformer forcluster"
time="2024-12-05T17:50:31Z" level=info msg="0x40002ce690 subscribed to settings updates"
time="2024-12-05T17:50:31Z" level=debug msg="Successfully saved info of 1 clusters"
time="2024-12-05T17:50:41Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:50:41Z" level=debug msg="Successfully saved info of 1 clusters"
time="2024-12-05T17:51:01Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:01Z" level=debug msg="Successfully saved info of 1 clusters"
time="2024-12-05T17:51:21Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:21Z" level=debug msg="Successfully saved info of 1 clusters"
time="2024-12-05T17:51:41Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:41Z" level=debug msg="Successfully saved info of 1 clusters"
time="2024-12-05T17:51:48Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:48Z" level=debug msg="Cluster https://kubernetes.default.svc has not changed shard"
time="2024-12-05T17:51:48Z" level=info msg="Refreshing app status (spec.source differs), level (3)" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:48Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: )" application=argocd/cert-manager-crds
time="2024-12-05T17:51:48Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:48Z" level=info msg="Start syncing cluster" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ApplicationSet.argoproj.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PersistentVolumeClaim on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PodDisruptionBudget.policy on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch RuntimeClass.node.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch NetworkPolicy.networking.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch StorageClass.storage.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch IngressClass.networking.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CSIDriver.storage.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CertificateSigningRequest.certificates.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Job.batch on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ReplicationController on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CronJob.batch on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PodTemplate on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Ingress.networking.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ValidatingWebhookConfiguration.admissionregistration.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch LimitRange on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CSIStorageCapacity.storage.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch HorizontalPodAutoscaler.autoscaling on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch VolumeAttachment.storage.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ResourceQuota on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch MutatingWebhookConfiguration.admissionregistration.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PersistentVolume on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch DaemonSet.apps on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch StatefulSet.apps on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CSINode.storage.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch AppProject.argoproj.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Application.argoproj.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Node on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PriorityClass.scheduling.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Role.rbac.authorization.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Deployment.apps on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch RoleBinding.rbac.authorization.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch APIService.apiregistration.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch PriorityLevelConfiguration.flowcontrol.apiserver.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch FlowSchema.flowcontrol.apiserver.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch CustomResourceDefinition.apiextensions.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Service on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ConfigMap on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Namespace on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ServiceAccount on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ControllerRevision.apps on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ClusterRoleBinding.rbac.authorization.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch EndpointSlice.discovery.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ClusterRole.rbac.authorization.k8s.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Endpoints on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch ReplicaSet.apps on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Pod on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Cluster successfully synced" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=info msg="Start watch Secret on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:48Z" level=debug msg="Generating Manifest for source {https://github.com/hlastras/test-argocd-crds.git services/cert-manager/multi-file HEAD nil nil nil nil } revision HEAD"
time="2024-12-05T17:51:50Z" level=info msg="GetRepoObjs stats" application=argocd/cert-manager-crds build_options_ms=0 manifests_ms=2347 plugins_ms=0 repo_ms=0 time_ms=2348
time="2024-12-05T17:51:50Z" level=debug msg="Retrieved live manifests" application=argocd/cert-manager-crds
time="2024-12-05T17:51:50Z" level=debug msg=app.status.expired application=argocd/cert-manager-crds useDiffCache=false
time="2024-12-05T17:51:50Z" level=debug msg="Finished getting app hosts" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds iterate_resources_ms=0 process_app_pods_by_node_ms=0 project=default time_ms=0
time="2024-12-05T17:51:50Z" level=debug msg="Finished getting resource tree" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_app_hosts_ms=0 get_app_proj_ms=0 get_orphaned_resources_ms=0 process_managed_resources_ms=5 process_orphaned_resources_ms=0 project=default time_ms=5
time="2024-12-05T17:51:50Z" level=debug msg="Finished setting app managed resources" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_resource_tree_ms=5 hide_secret_data_ms=3 project=default set_app_managed_resources_ms=30 set_app_resources_tree_ms=1 time_ms=41
time="2024-12-05T17:51:50Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:50Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:50Z" level=info msg="Initiated automated sync to 'ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54'" application=cert-manager-crds dest-namespace= dest-server="https://kubernetes.default.svc" reason=OperationStarted type=Normal
time="2024-12-05T17:51:50Z" level=info msg="Initiated automated sync to 'ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54'" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:50Z" level=debug msg="Finished auto sync" already_attempted_check_ms=0 already_attempted_sync_ms=0 app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_applications_ms=0 project=default set_app_operation_ms=17 time_ms=44 write_back_to_informer_ms=0
time="2024-12-05T17:51:50Z" level=info msg="Updated sync status: -> OutOfSync" application=cert-manager-crds dest-namespace= dest-server="https://kubernetes.default.svc" reason=ResourceUpdated type=Normal
time="2024-12-05T17:51:50Z" level=info msg="Updated health status: -> Missing" application=cert-manager-crds dest-namespace= dest-server="https://kubernetes.default.svc" reason=ResourceUpdated type=Normal
time="2024-12-05T17:51:50Z" level=info msg="Update successful" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:50Z" level=info msg="Reconciliation completed" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds app_status_update_ms=0 application=cert-manager-crds auto_sync_ms=44 compare_app_state_ms=2387 comparison-level=3 comparison_with_nothing_ms=0 dedup_ms=0 dest-name= dest-namespace= dest-server="https://kubernetes.default.svc" diff_ms=38 git_ms=2348 health_ms=0 live_ms=0 normalize_application_ms=0 patch_ms=8 persist_app_status_ms=20 process_finalizers_ms=0 project=default refresh_app_conditions_ms=0 set_app_managed_resources_ms=41 setop_ms=17 settings_ms=0 sync_ms=0 time_ms=2494
time="2024-12-05T17:51:50Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:50Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:50Z" level=info msg="Start Update application operation state"
time="2024-12-05T17:51:50Z" level=info msg="Completed Update application operation state"
time="2024-12-05T17:51:50Z" level=info msg="updated 'argocd/cert-manager-crds' operation (phase: Running)" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:50Z" level=info msg="Initialized new operation: {&SyncOperation{Revision:ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54,Prune:false,DryRun:false,SyncStrategy:nil,Resources:]SyncOperationResource{},Source:nil,Manifests:],SyncOptions:[ServerSideApply=true],Sources:]ApplicationSource{},Revisions:],SelfHealAttemptsCount:0,} { true} ] {5 nil}}" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:50Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: )" application=argocd/cert-manager-crds
time="2024-12-05T17:51:50Z" level=debug msg="Generating Manifest for source {https://github.com/hlastras/test-argocd-crds.git services/cert-manager/multi-file HEAD nil nil nil nil } revision ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54"
time="2024-12-05T17:51:50Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:50Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:50Z" level=info msg="GetRepoObjs stats" application=argocd/cert-manager-crds build_options_ms=0 manifests_ms=67 plugins_ms=0 repo_ms=0 time_ms=67
time="2024-12-05T17:51:50Z" level=debug msg="Retrieved live manifests" application=argocd/cert-manager-crds
time="2024-12-05T17:51:50Z" level=debug msg="using diff cache" application=argocd/cert-manager-crds useDiffCache=true
time="2024-12-05T17:51:50Z" level=info msg=Syncing application=argocd/cert-manager-crds skipHooks=false started=false syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg="Tasks from managed resources" application=argocd/cert-manager-crds resourceTasks="[Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,)]" syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg="tasks from hooks" application=argocd/cert-manager-crds hookTasks="]" syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg="Tasks (dry-run)" application=argocd/cert-manager-crds syncId=00001-dbUpA tasks="[Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)]"
time="2024-12-05T17:51:50Z" level=info msg="Running tasks" application=argocd/cert-manager-crds dryRun=true numTasks=6 syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/certificaterequests.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/clusterissuers.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/certificates.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/challenges.acme.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/issuers.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=true syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/orders.acme.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=client manager=argocd-controller serverSideApply=false serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg="Filtering out non-pending tasks" application=argocd/cert-manager-crds syncId=00001-dbUpA tasks="[Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)]"
time="2024-12-05T17:51:50Z" level=info msg="Filtering tasks in correct phase and wave" application=argocd/cert-manager-crds phase=Sync syncFailTasks="]" syncId=00001-dbUpA tasks="[Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)]" wave=0
time="2024-12-05T17:51:50Z" level=info msg="Updating operation state. phase: Running -> Running, message: '' -> 'one or more tasks are running'" application=argocd/cert-manager-crds syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg=Wet-run application=argocd/cert-manager-crds syncId=00001-dbUpA tasks="[Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,), Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)]"
time="2024-12-05T17:51:50Z" level=info msg="Running tasks" application=argocd/cert-manager-crds dryRun=false numTasks=6 syncId=00001-dbUpA
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificaterequests.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/certificaterequests.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/clusterissuers.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/orders.acme.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/orders.acme.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/clusterissuers.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/certificates.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/challenges.acme.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg=Applying application=argocd/cert-manager-crds dryRun=false syncId=00001-dbUpA task="Sync/0 resource apiextensions.k8s.io/CustomResourceDefinition:/issuers.cert-manager.io nil->obj (,,)"
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/certificates.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/challenges.acme.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:50Z" level=info msg="Applying resource CustomResourceDefinition/issuers.cert-manager.io in cluster: https://10.96.0.1:443, namespace: " dry-run=none manager=argocd-controller serverSideApply=true serverSideDiff=false
time="2024-12-05T17:51:51Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=certificaterequests.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=orders.acme.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=certificates.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=issuers.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Start watch Order.acme.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Start watch CertificateRequest.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Start watch Certificate.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=challenges.acme.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Start watch Issuer.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:51Z" level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=clusterissuers.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Updating operation state. phase: Running -> Succeeded, message: 'one or more tasks are running' -> 'successfully synced (all tasks run)'" application=argocd/cert-manager-crds syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="sync/terminate complete" application=argocd/cert-manager-crds duration=1.063998386s syncId=00001-dbUpA
time="2024-12-05T17:51:51Z" level=info msg="Start Update application operation state"
time="2024-12-05T17:51:51Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:51Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:51Z" level=info msg="Completed Update application operation state"
time="2024-12-05T17:51:51Z" level=info msg="updated 'argocd/cert-manager-crds' operation (phase: Succeeded)" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:51Z" level=info msg="Sync operation to ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54 succeeded" application=cert-manager-crds dest-namespace= dest-server="https://kubernetes.default.svc" reason=OperationCompleted type=Normal
time="2024-12-05T17:51:51Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:51Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:51Z" level=debug msg="Finished processing requested app operation" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds final_set_operation_state=20 get_app_proj_ms=0 initial_operation_stage_ms=10 project=default request_app_refresh_ms=0 time_ms=1175 validate_and_sync_app_state_ms=1144
time="2024-12-05T17:51:51Z" level=debug msg="Finished processing app operation queue item" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_fresh_app_ms=6 process_requested_app_operation_ms=1175 project=default time_ms=1181
time="2024-12-05T17:51:51Z" level=debug msg="Finished processing app operation queue item" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_fresh_app_ms=0 project=default time_ms=0
time="2024-12-05T17:51:51Z" level=info msg="Refreshing app status (controller refresh requested), level (3)" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:51Z" level=info msg="Comparing app state (cluster: https://kubernetes.default.svc, namespace: )" application=argocd/cert-manager-crds
time="2024-12-05T17:51:51Z" level=debug msg="Generating Manifest for source {https://github.com/hlastras/test-argocd-crds.git services/cert-manager/multi-file HEAD nil nil nil nil } revision HEAD"
time="2024-12-05T17:51:52Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=info msg="Start watch Challenge.acme.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=info msg="Start watch ClusterIssuer.cert-manager.io on https://10.96.0.1:443" server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=info msg="GetRepoObjs stats" application=argocd/cert-manager-crds build_options_ms=0 manifests_ms=525 plugins_ms=0 repo_ms=0 time_ms=525
time="2024-12-05T17:51:52Z" level=debug msg="Retrieved live manifests" application=argocd/cert-manager-crds
time="2024-12-05T17:51:52Z" level=debug msg="using diff cache" application=argocd/cert-manager-crds useDiffCache=true
time="2024-12-05T17:51:52Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=debug msg="Finished getting app hosts" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds iterate_resources_ms=3 process_app_pods_by_node_ms=0 project=default time_ms=3
time="2024-12-05T17:51:52Z" level=debug msg="Finished getting resource tree" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_app_hosts_ms=3 get_app_proj_ms=0 get_orphaned_resources_ms=0 process_managed_resources_ms=12 process_orphaned_resources_ms=0 project=default time_ms=16
time="2024-12-05T17:51:52Z" level=debug msg="Finished setting app managed resources" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_resource_tree_ms=16 hide_secret_data_ms=4 project=default set_app_managed_resources_ms=27 set_app_resources_tree_ms=1 time_ms=48
time="2024-12-05T17:51:52Z" level=info msg="Executing compare of syncResult.Revision and commitSha because manifest changed: ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54" application=cert-manager-crds
time="2024-12-05T17:51:52Z" level=info msg="Skipping auto-sync: most recent sync already to ff98591d1e0392bb4aa5fad2c5a4ad692efc5a54" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:52Z" level=debug msg="Finished auto sync" already_attempted_sync_ms=0 app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default time_ms=0
time="2024-12-05T17:51:52Z" level=info msg="Update successful" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds project=default
time="2024-12-05T17:51:52Z" level=info msg="Reconciliation completed" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds app_status_update_ms=0 application=cert-manager-crds auto_sync_ms=0 compare_app_state_ms=959 comparison-level=3 comparison_with_nothing_ms=0 dedup_ms=0 dest-name= dest-namespace= dest-server="https://kubernetes.default.svc" diff_ms=62 git_ms=525 health_ms=0 live_ms=370 normalize_application_ms=1 patch_ms=11 persist_app_status_ms=11 process_finalizers_ms=0 project=default refresh_app_conditions_ms=0 set_app_managed_resources_ms=48 setop_ms=0 settings_ms=0 sync_ms=0 time_ms=1021
time="2024-12-05T17:51:52Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:51:52Z" level=debug msg="Skipping sharding distribution update. No relevant changes"
time="2024-12-05T17:51:52Z" level=debug msg="Finished processing app operation queue item" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_fresh_app_ms=0 project=default time_ms=0
time="2024-12-05T17:51:52Z" level=debug msg="Finished processing app operation queue item" app-namespace=argocd app-qualified-name=argocd/cert-manager-crds application=cert-manager-crds get_fresh_app_ms=0 project=default time_ms=0
time="2024-12-05T17:51:52Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:52Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=ADDED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:53Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:51:54Z" level=info msg="Updating Kubernetes APIs, watches, and Open API schemas due to CRD event" eventType=MODIFIED groupKind=CustomResourceDefinition.apiextensions.k8s.io server="https://kubernetes.default.svc"
time="2024-12-05T17:52:01Z" level=debug msg="Checking if cluster https://kubernetes.default.svc with clusterShard 0 should be processed by shard 0"
time="2024-12-05T17:52:01Z" level=debug msg="Successfully saved info of 1 clusters"
```

**Extra context**

We experience this issue when creating new clusters and need to deploy a set of applications at the beginning. We always have to resolve this manually.
Our ArgoCD setup is more complex (with many non-default configurations). Additionally, our Applications are more advanced, and we use different SyncOptions. However, testing this, I found that even when removing all our customizations and simplifying it as much as possible, the issue still occurs.

Looking at the logs, I would say that the resources marked as OutOfSync are actually shown as deployed in the logs:
```
level=info msg="Adding resource result, status: 'Synced', phase: 'Running', message: 'customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io serverside-applied'" application=argocd/cert-manager-crds kind=CustomResourceDefinition name=challenges.acme.cert-manager.io namespace= phase=Sync syncId=00001-dbUpA
```
Maybe I'm mistaken on that log.

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the provided kind create cluster command, Helm installation, and cert-manager-crds Application manifest. Start with the application-controller debug logs while deploying multiple CRDs, then compare the reported resource state before and after a manual refresh or the three-minute resync. Done means successfully deployed CRDs leave the Application Healthy and Synced without manual synchronization.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, helm, kubernetes
Domain
ci-cd, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.