nextcloud using hpa deadlock

Hi there,

I've setup PVC using "ReadWriteMany" accessMode for the nextcloud data directory over NFS.
The NFS share is exposed by TrueNas and using democratic-csi driver for k8s: https://jonathangazeley.com/2021/01/05/using-truenas-to-provide-persistent-storage-for-kubernetes/

It seems while configuring nextcloud with the following values using multiple replicas:

replicaCount: 3
hpa:
  cputhreshold: 60
  enabled: true
  maxPods: 10
  minPods: 3

That there's a kind of deadlock between pods waiting each other releasing a lock file:

+ [ ! -f /var/www/html/nextcloud-init-sync.lock ]                                                                                                                                                                         
+ count=2
+ wait=20
+ echo Another process is initializing Nextcloud. Waiting 20 seconds...
+ sleep 20
Another process is initializing Nextcloud. Waiting 20 seconds...

Which never happened and pods after a while entering in a CrashLoopBack state and this preventing nextcloud to start properly.

My understanding is that the first pod launch will acquired the lock preventing the other pods to sync the html folder, one this pod finished syncing the html folder the initialization is assumed to be complete, would it be possible that this rsync task takes to much times are finish and times out?

I don't know if that rings a bell to someone ?

Cheers.

Start with the Helm values for replicaCount and hpa, then inspect the Nextcloud initialization path involving /var/www/html/nextcloud-init-sync.lock and the rsync task. Reproduce with three replicas on the described RWX NFS setup, determine why the lock is not released before the wait expires, and verify that the pods start without CrashLoopBackOff.