Readiness/Liveness failing while healthz check on celery-celery

Describe the issue

I've split helm chart configuration to the as follow:
weblate-web.yaml

---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: weblate-web
  namespace: weblate
spec:
  interval: 30m
  chart:
    spec:
      chart: weblate
      version: '0.5.9'
      sourceRef:
        kind: HelmRepository
        name: weblate
        namespace: flux-system
      interval: 24h
  values:
    replicaCount: 1
    labels:
      app: "weblate-web"
      flux: "true"
      helm: "true"
    existingSecret: "weblate"
    allowedHosts: "*"
    debug: "1"
    externalSecretName: "weblate-configs"
    extraConfig:
      WEBLATE_LOGLEVEL: "DEBUG"
      WEBLATE_UPDATE_LANGUAGES: "0"
      WEBLATE_ENABLE_HTTPS: "1"
      WEBLATE_REGISTRATION_OPEN: "0"
      WEBLATE_ENABLE_SHARING: "0"
      WEBLATE_SUPPORT_STATUS_CHECK: "0"
      WEBLATE_REQUIRE_LOGIN: "0"
      WEBLATE_NO_EMAIL_AUTH: "1"
      WEBLATE_REGISTRATION_ALLOW_BACKENDS: "google-oauth2"
      WEBLATE_SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: "some.host"
      WEBLATE_GITLAB_HOST: "gitlab.some.host"
      WEBLATE_AUTO_UPDATE: "1"
      WEBLATE_SERVICE: "web"
    siteTitle: "My Awesome Weblate"
    siteDomain: "weblate.some.host"
    serviceAccount:
      create: false
    podSecurityContext:
      enabled: true
      fsGroup: 1000
    podAnnotations: {}
    containerSecurityContext:
      enabled: false
    service:
      type: ClusterIP
      port: 80
    ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: "nginx"
      hosts:
        - host: celery.some.host
          paths:
            - path: /
              pathType: Prefix
    persistence:
      enabled: true
      accessMode: ReadWriteOnce
      size: 10Gi
      filestore_dir: /app/data
    resources:
      limits:
        cpu: 500m
        memory: 1024Mi
      requests:
        cpu: 500m
        memory: 1024Mi
    nodeSelector: {}
    tolerations: []
    affinity: {}
    livenessProbe:
      initialDelaySeconds: 300
      periodSeconds: 30
      failureThreshold: 10
    readinessProbe:
      initialDelaySeconds: 60
      periodSeconds: 30
      failureThreshold: 2
    postgresql:
      enabled: false
      postgresqlHost: postgres.some.host
    redis:
      auth:
        enabled: false
      enabled: false
      redisHost: redis.some.host

weblate-celery.yaml

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: weblate-celery
  namespace: weblate
spec:
  interval: 30m
  chart:
    spec:
      chart: weblate
      version: '0.5.9'
      sourceRef:
        kind: HelmRepository
        name: weblate
        namespace: flux-system
      interval: 24h
  values:
    replicaCount: 1
    labels:
      app: "weblate-celery"
      flux: "true"
      helm: "true"
    existingSecret: "weblate"
    allowedHosts: "*"
    debug: "1"
    externalSecretName: "weblate-configs"
    extraConfig:
      WEBLATE_LOGLEVEL: "DEBUG"
      WEBLATE_UPDATE_LANGUAGES: "0"
      WEBLATE_ENABLE_HTTPS: "1"
      WEBLATE_REGISTRATION_OPEN: "0"
      WEBLATE_ENABLE_SHARING: "0"
      WEBLATE_SUPPORT_STATUS_CHECK: "0"
      WEBLATE_REQUIRE_LOGIN: "0"
      WEBLATE_NO_EMAIL_AUTH: "1"
      WEBLATE_REGISTRATION_ALLOW_BACKENDS: "google-oauth2"
      WEBLATE_SOCIAL_AUTH_GOOGLE_OAUTH2_WHITELISTED_DOMAINS: "some.host"
      WEBLATE_GITLAB_HOST: "gitlab.some.host"
      WEBLATE_AUTO_UPDATE: "1"
      WEBLATE_SERVICE: "celery-celery"
    siteTitle: "My Awesome Weblate"
    siteDomain: "weblate.some.host"
    serviceAccount:
      create: false
    podSecurityContext:
      enabled: true
      fsGroup: 1000
    podAnnotations: {}
    containerSecurityContext:
      enabled: false
    service:
      type: ClusterIP
      port: 80
    ingress:
      enabled: true
      annotations:
        kubernetes.io/ingress.class: "nginx"
      hosts:
        - host: celery.some.host
          paths:
            - path: /
              pathType: Prefix
    persistence:
      enabled: true
      existingClaim: "weblate-web"
      accessMode: ReadWriteOnce
      size: 10Gi
      filestore_dir: /app/data
    resources:
      limits:
        cpu: 700m
        memory: 1024Mi
      requests:
        cpu: 700m
        memory: 1024Mi
    nodeSelector: {}
    tolerations: []
    affinity: {}
    livenessProbe:
      initialDelaySeconds: 300
      periodSeconds: 30
      failureThreshold: 10
    readinessProbe:
      initialDelaySeconds: 60
      periodSeconds: 30
      failureThreshold: 2
    postgresql:
      enabled: false
      postgresqlHost: postgres.some.host
    redis:
      auth:
        enabled: false
      enabled: false
      redisHost: redis.some.host

But while kybernates try to check Readiness/Liveness probe healthz pod with WEBLATE_SERVICE=celery-celery , it always receive

Liveness probe failed: Get "http://{my_pod_ip}:8080/healthz/": dial tcp {my_pod_ip}:8080: connect: connection refused

In this time while checking WEBLATE_SERVICE=web- response code 200.

I know that i can use some like this in part of Probe configuration, instead of httpGet:

...
exec:
  command: ["celery", "--app=weblate.utils", "inspect", "ping"]
...

Unfortunately my flux server not support custom probe setup - only default httpGet

Also i tried to use my own http server for pod with WEBLATE_SERVICE=celery-celery, that i run inside settings-override.py - but it's not clear way.

I already tried

• I've read and searched the documentation.
• I've searched for similar filed issues in this repository.

Steps to reproduce the behavior

1. Setup weblate helm release chart like i described upper
2. Try to check pod with WEBLATE_SERVICE: "celery-celery" url {my_pod_ip}:8080/healthz/
3. Always getting

Liveness probe failed: Get "http://{my_pod_ip}:8080/healthz/": dial tcp {my_pod_ip}:8080: connect: connection refused

Expected behavior

No response

Screenshots

No response

Exception traceback

No response

Additional context

No response

The issue provides separate HelmRelease values for web and celery, including WEBLATE_SERVICE and the HTTP probes, but names no repository file or test. Start by rendering the chart templates for the liveness/readiness probes and compare the generated workload for web with celery. Done means the celery workload has a supported health check that does not receive connection refused, with coverage for the rendered configuration.