apache / apache/openwhisk-deploy-kube

Challenges encountered when deploying to DIY K8s Cluster

Open
#775 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
309
Forks
231
PR merge metrics
No merged PRs in 30d

Description

Deploying Apache OpenWhisk on a DIY Kubernetes cluster can throw a couple of curveballs. Here's a quick rundown of the snags I hit and how I untangled them.

1. **GitHub cloning hiccups in `initCouchDB` and `installPackages`**

**The Problem:** Due to network issues, the setup scripts couldn't clone from GitHub.

**The Fix:** Tweak the scripts `initdb.sh` and `myTask.sh` under `helm/openwhisk/configMapFiles/initCouchDB/` to point to a mirror instead of `github.com/apache/openwhisk`.

2. **JVM Bus Error in Controller and Invoker Pods**

**The Problem:** JVM threw a bus error raised by JIT compilation, crashing both the controller and invoker pods.

**The Fix:** Disable JIT and boost the heap size to 16GB to stabilize the JVM.

```yaml
# mycluster.yaml
controller:
jvmHeapMB: "16384"
jvmOptions: "-Xint"

invoker:
jvmHeapMB: "16384"
jvmOptions: "-Xint"
```

3. **Probes for controller getting axed by Kubernetes**

**The Problem:** Kubernetes was terminating controller pod for taking too long, as we disabled JIT.

**The Fix:** Set the readiness probe timeout to a generous 180 seconds to avoid premature termination.

```yaml
# mycluster.yaml
probes:
controller:
livenessProbe:
initialDelaySeconds: 180
periodSeconds: 180
timeoutSeconds: 10
readinessProbe:
initialDelaySeconds: 180
periodSeconds: 180
timeoutSeconds: 10
```

4. **Custom Kubernetes DNS Configuration**

**The Problem:** Needed to align DNS settings with DIY cluster setup, otherwise will fail Nginx pod.

**The Fix:** Specify the DNS service (RKE2 in my case) in the cluster config.

```yaml
# mycluster.yaml
k8s:
dns: rke2-coredns-rke2-coredns.kube-system
```

Contributor guide

Open the contributing guide

Research direction

Start by reading helm/openwhisk/configMapFiles/initCouchDB/initdb.sh and myTask.sh, then inspect the mycluster.yaml settings described in the issue. Confirm whether these DIY Kubernetes fixes are already documented; done means adding a clear, repository-appropriate record of the relevant configuration and deployment findings.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, shell
Domain
devops, infrastructure
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.