apache / apache/openwhisk-deploy-kube

Incomplete deployment with private Docker registry

Open
#721 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
309
Forks
231
PR merge metrics
No merged PRs in 30d

Description

Hi!

I've been following the documentation on [how to set up OpenWhisk with a private Docker registry](https://github.com/apache/openwhisk-deploy-kube/blob/master/docs/private-docker-registry.md). After configuring authorization with a self-signed certificate, which works fine, and finding out all versions of images that need to be transported to the private registry, I got the deployment with `helm` almost ready. Everything seems to be deployed except for invoker pods:

```
openwhisk owdev-alarmprovider-85c6cb4f6d-gsfnz 1/1 Running 0 3m54s
openwhisk owdev-apigateway-64c77ddb4-2gvr8 1/1 Running 0 3m54s
openwhisk owdev-controller-0 1/1 Running 0 3m54s
openwhisk owdev-couchdb-d75b8cf5c-xlxz5 1/1 Running 0 3m54s
openwhisk owdev-gen-certs-6tpbz 0/1 Completed 0 3m54s
openwhisk owdev-init-couchdb-7cqtq 0/1 Completed 0 3m54s
openwhisk owdev-install-packages-fhg2l 0/1 Init:0/1 0 3m54s
openwhisk owdev-invoker-0 1/1 Running 0 3m54s
openwhisk owdev-kafka-0 1/1 Running 0 3m54s
openwhisk owdev-kafkaprovider-696dcc45f9-24m4c 1/1 Running 0 3m54s
openwhisk owdev-nginx-65775cb5d6-zzsng 1/1 Running 0 3m54s
openwhisk owdev-redis-7775bb848f-f2twc 1/1 Running 0 3m54s
openwhisk owdev-seccomp-6q9cd 1/1 Running 0 3m54s
openwhisk owdev-seccomp-n5x8m 1/1 Running 0 3m54s
openwhisk owdev-wskadmin 1/1 Running 0 3m54s
openwhisk owdev-zookeeper-0 1/1 Running 0 3m54s
openwhisk wskowdev-invoker-00-6-whisksystem-invokerhealthtestaction0 0/1 ErrImagePull 0 40s
openwhisk wskowdev-invoker-00-7-prewarm-nodejs14 0/1 ErrImagePull 0 39s
```

A closer inspection suggests that pods are not authorized to access the registry and it does not look like any certificate and network availability issue, but an incorrect/missing authorization:

```
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 31s default-scheduler Successfully assigned openwhisk/wskowdev-invoker-00-1-prewarm-nodejs14 to kind-worker2
Normal Pulling 16s (x2 over 30s) kubelet Pulling image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0"
Warning Failed 16s (x2 over 30s) kubelet Failed to pull image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": rpc error: code = Unknown desc = failed to pull and unpack image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": failed to resolve reference "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0": unexpected status code [manifests 1.19.0]: 401 Unauthorized
Warning Failed 16s (x2 over 30s) kubelet Error: ErrImagePull
Normal BackOff 3s (x2 over 29s) kubelet Back-off pulling image "192.168.0.19:5000/openwhisk/action-nodejs-v14:1.19.0"
Warning Failed 3s (x2 over 29s) kubelet Error: ImagePullBackOff
```

I'm not using any custom images and I have not even tried to create any actions - this is also from standard deployment. The image is available in my registry. Does it look like there's a missing docker login on these pods?

I am using OpenWhisk with helm 3.8 and kind 0.11.

Contributor guide

Open the contributing guide

Research direction

Start with docs/private-docker-registry.md and the Helm deployment configuration for invoker-created action pods. Reproduce the deployment with Helm 3.8 on kind 0.11, then inspect the pod events and registry authorization settings. Done means the invoker health-test and prewarm Node.js 14 images pull successfully from the private registry.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, helm, kubernetes
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.