OSMO backend-listener stuck in init container if fetch APKINDEX fails

Describe the bug.

Observation:

When deploying OSMO localy on nvkind, if in the init phase of the backend-listener fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz fails, than the init container gets stuck.

Suggestion:

Fail the init container or use a container that does not need to select packages.

Steps:

1. Follow the local install guide from main up to step 3
2. Install Osmo using helm and the chart quick-start-1.0.0.tgz

The command in the documentation is:

$ helm fetch https://helm.ngc.nvidia.com/nvidia/osmo/charts/quick-start-1.0.0.tgz
$ helm upgrade --install osmo quick-start-1.0.0.tgz \
  --namespace osmo \
  --create-namespace \
  --wait

Details:
A pods corresponding to OSMO's components span and initalize. The backend listener is stuck in the init phase:

NAME                                          READY   STATUS     RESTARTS   AGE
localstack-s3-856897f5ff-fkld8                1/1     Running    0          29m
osmo-agent-5886cd89b5-vfxjq                   1/1     Running    0          29m
osmo-delayed-job-monitor-5fdcc466c9-f5sck     1/1     Running    0          29m
osmo-logger-65f978bb55-xpt7l                  1/1     Running    0          29m
osmo-osmo-backend-listener-65d8d84c84-xxwfd   0/1     Init:1/2   0          29m
osmo-osmo-backend-worker-6c585b9679-z5pt9     1/1     Running    0          29m
osmo-router-56485f9b9b-kfgg8                  1/1     Running    0          29m
osmo-service-6bd797b596-n8s48                 1/1     Running    0          29m
osmo-ui-7f546f6f95-kj56x                      1/1     Running    0          29m
osmo-worker-778d8b49db-t8t5h                  1/1     Running    0          29m
postgres-856869c8cc-bqj5k                     1/1     Running    0          29m
quick-start-76f65c74f8-d2w28                  1/1     Running    0          29m
redis-694898cfdf-65fl2                        1/1     Running    0          29m

kubectl describe yields the following:

...
 Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  26m   default-scheduler  Successfully assigned osmo/osmo-osmo-backend-listener-65d8d84c84-xxwfd to osmo-worker4
  Normal  Pulling    26m   kubelet            Pulling image "busybox:1.37.0"
  Normal  Pulled     26m   kubelet            Successfully pulled image "busybox:1.37.0" in 776ms (4.407s including waiting). Image size: 2224358 bytes.
  Normal  Created    26m   kubelet            Created container: wait-for-ingress
  Normal  Started    26m   kubelet            Started container wait-for-ingress
  Normal  Pulling    24m   kubelet            Pulling image "alpine:3.18"
  Normal  Pulled     24m   kubelet            Successfully pulled image "alpine:3.18" in 1.167s (4.749s including waiting). Image size: 3428098 bytes.
  Normal  Created    24m   kubelet            Created container: wait-for-token
  Normal  Started    24m   kubelet            Started container wait-for-token

Inspecting the init container shows:

fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
WARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/main: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
WARNING: fetching https://dl-cdn.alpinelinux.org/alpine/v3.18/community: temporary error (try again later)
ERROR: unable to select packages:
  curl (no such package):
    required by: world[curl]
  jq (no such package):
    required by: world[jq]
Waiting for backend-operator-token to be created...
sh: curl: not found
Token backend-operator-token not found, waiting...
sh: curl: not found
Token backend-operator-token not found, waiting...
sh: curl: not found
Token backend-operator-token not found, waiting...
sh: curl: not found
Token backend-operator-token not found, waiting...
sh: curl: not found
...

deleting the pod which leads to restart fixes the problem. It was likely a temporal unavailability.

Environment

• Kubernetes version:
  kind version 0.30.0

• OSMO version:
  OSMO client version: 6.0.0.8fad4ecd8
  OSMO service version: 6.0.0.8fad4ecd8

• Cloud provider or hardware configuration:
  local, nvkind + kind

Code of Conduct

• I agree to follow NVIDIA OSMO's Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report

Start by inspecting the backend-listener init-container definition in the quick-start Helm chart and its wait-for-token command, then reproduce the failed APKINDEX fetch during the local install guide steps. Done means a transient package-index failure cannot leave the pod indefinitely reporting missing curl and jq: the init container either exits for restart or uses an available package-independent approach.