argoproj / argoproj/argo-cd

Argo CD stuck indefinitely while refreshing an App

Open
#15,519 12 comments 7 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
24.2k
Forks
7.8k
Avg merge
1d 12h
Merged PRs (30d)
234

Description

Checklist:

* [ Y] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
* [ Y] I've included steps to reproduce the bug.
* [ Y] I've pasted the output of `argocd version`.

**Describe the bug**

We are getting issues on multiple EKS clusters where Apps get stuck in refreshing state indefinitely.

**To Reproduce**
1. Enable Auto Sync.
2. Deploy a new App.

**Expected behavior**
App will get stuck in refreshing state indefinitely.
Even when we try to use command argocd app terminate-op APP-NAME, it doesn't help.
**Screenshots**

**Version**
v2.5.1

**Logs**

In logs we found that it times out after 3 mins, then again it retries to refresh the app status and again fails. This keeps on happening in loop infinitely.:
time="2023-09-15T08:04:19Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:04:04Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:03:19Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:03:04Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:04:34Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:05:05Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:05:20Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:05:35Z" level=info msg="Refreshing app status (controller refresh requested), level (1)" application=argocd/XXXX
time="2023-09-15T08:06:05Z" level=info msg="Refreshing app status (comparison expired, requesting refresh. reconciledAt: 2023-09-15 08:03:04 +0000 UTC, expiry: 3m0s), level (2)" application=argocd/XXXX

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with Auto Sync enabled and a newly deployed App, then inspect the controller logs around the three-minute refresh timeout and retry loop. Use `argocd app terminate-op APP-NAME` during reproduction. Done means the App can leave the refreshing state and refresh operations no longer loop indefinitely.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go, kubernetes
Domain
ci-cd, cloud, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.