apache / apache/openwhisk-deploy-kube
Incomplete deployment with private Docker registry
- Dominant language
- Shell
- Stars
- 309
- Forks
- 231
- PR merge metrics
- No merged PRs in 30d
Description
Hi!
I've been following the documentation on [how to set up OpenWhisk with a private Docker registry](https://github.com/apache/openwhisk-deploy-kube/blob/master/docs/private-docker-registry.md). After configuring authorization with a self-signed certificate, which works fine, and finding out all versions of images that need to be transported to the private registry, I got the deployment with `helm` almost ready. Everything seems to be deployed except for invoker pods:
```
openwhisk owdev-alarmprovider-85c6cb4f6d-gsfnz 1/1 Running 0 3m54s
openwhisk owdev-apigateway-64c77ddb4-2gvr8 1/1 Running 0 3m54s
openwhisk owdev-controller-0 1/1 Running 0 3m54s
openwhisk owdev-couchdb-d75b8cf5c-xlxz5 1/1 Running 0 3m54s
openwhisk owdev-gen-certs-6tpbz 0/1 Completed 0 3m54s
openwhisk owdev-init-couchdb-7cqtq 0/1 Completed 0 3m54s
openwhisk owdev-install-packages-fhg2l 0/1 Init:0/1 0 3m54s
openwhisk owdev-invoker-0 1/1 Running 0 3m54s
openwhisk owdev-kafka-0 1/1 Running 0 3m54s
openwhisk owdev-kafkaprovider-696dcc45f9-24m4c 1/1 Running 0 3m54s
openwhisk owdev-nginx-65775cb5d6-zzsng 1/1 Running 0 3m54s
openwhisk owdev-redis-7775bb848f-f2twc 1/1 Running 0 3m54s
openwhisk owdev-seccomp-6q9cd 1/1 Running 0 3m54s
openwhisk owdev-seccomp-n5x8m 1/1 Running 0 3m54s
openwhisk owdev-wskadmin 1/1 Running 0 3m54s
openwhisk owdev-zookeeper-0 1/1 Running 0 3m54s
openwhisk wskowdev-invoker-00-6-whisksystem-invokerhealthtestaction0 0/1 ErrImagePull 0 40s
openwhisk wskowdev-invoker-00-7-prewarm-nodejs14 0/1 ErrImagePull 0 39s
```
A closer inspection suggests that pods are not authorized to access the registry and it does not look like any certificate and network availability issue, but an incorrect/missing authorization:
```
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 31s default-scheduler Successfully assigned openwhisk/wskowdev-invoker-00-1-prewarm-nodejs14 to kind-worker2
Normal Pulling 16s (x2 over 30s) kubelet Pulling image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0"
Warning Failed 16s (x2 over 30s) kubelet Failed to pull image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": rpc error: code = Unknown desc = failed to pull and unpack image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": failed to resolve reference "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": unexpected status code [manifests 1.19.0]: 401 Unauthorized
Warning Failed 16s (x2 over 30s) kubelet Error: ErrImagePull
Normal BackOff 3s (x2 over 29s) kubelet Back-off pulling image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0"
Warning Failed 3s (x2 over 29s) kubelet Error: ImagePullBackOff
```
I'm not using any custom images and I have not even tried to create any actions - this is also from standard deployment. The image is available in my registry. Does it look like there's a missing docker login on these pods?
I am using OpenWhisk with helm 3.8 and kind 0.11.
Contributor guide
Research direction
Start with docs/private-docker-registry.md and the Helm deployment configuration for invoker-created action pods. Reproduce the deployment with Helm 3.8 on kind 0.11, then inspect the pod events and registry authorization settings. Done means the invoker health-test and prewarm Node.js 14 images pull successfully from the private registry.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, helm, kubernetes
- Domain
- cloud, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100